-- Logs begin at Thu 2022-05-26 12:59:23 UTC, end at Thu 2022-05-26 14:18:31 UTC. -- May 26 13:40:55.975136 dv-cinder-842130-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 13:40:55.983031 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 13:40:55.985026 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 13:40:55 2022] *** May 26 13:40:55.985026 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 13:40:55.985026 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 13:40:55.985026 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: nodename: dv-cinder-842130-1-iscsi May 26 13:40:55.985026 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: machine: x86_64 May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: clock source: unix May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: pcre jit disabled May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: detected number of CPU cores: 4 May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: current working directory: / May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: detected binary path: /usr/bin/uwsgi-core May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: your processes number limit is 47680 May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: your memory page size is 4096 bytes May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: detected max file descriptor number: 2048 May 26 13:40:55.985397 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: lock engine: pthread robust mutexes May 26 13:40:55.987561 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: thunder lock: enabled May 26 13:40:55.987561 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 13:40:55.987561 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 13:40:56.031731 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: Python main interpreter initialized at 0x5607872d49c0 May 26 13:40:56.031731 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: python threads support enabled May 26 13:40:56.031731 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: your server socket listen backlog is limited to 100 connections May 26 13:40:56.031731 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: your mercy for graceful operations on workers is 90 seconds May 26 13:40:56.032096 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: mapped 403077 bytes (393 KB) for 2 cores May 26 13:40:56.032172 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: *** Operational MODE: preforking *** May 26 13:40:56.032419 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: *** uWSGI is running in multiple interpreter mode *** May 26 13:40:56.032419 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: spawned uWSGI master process (pid: 80227) May 26 13:40:56.032624 dv-cinder-842130-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 13:40:56.037185 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: spawned uWSGI worker 1 (pid: 80236, cores: 1) May 26 13:40:56.037185 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: spawned uWSGI worker 2 (pid: 80237, cores: 1) May 26 13:40:56.037185 dv-cinder-842130-1-iscsi devstack@keystone.service[80227]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 13:41:01.659448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:41:01.659448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: from cryptography.utils import int_from_bytes May 26 13:41:01.660219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:41:01.660219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: from cryptography.utils import int_from_bytes May 26 13:41:01.694993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:41:01.694993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes May 26 13:41:01.695516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:41:01.695516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes May 26 13:41:01.709626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 13:41:01.709820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 13:41:01.710009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] command line args: [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 13:41:01.710276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 13:41:01.710451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ================================================================================ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 13:41:01.710701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] admin_token = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.710941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_dir = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.711125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.711302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_source = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.711513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] debug = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.711711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_config_append = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_dir = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.713876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.714123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.714368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.714611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.714788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.714965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.715204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.715447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.715677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.715906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.716137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.716366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.716615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.716858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.717129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.717640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.91/identity {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.717820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] publish_errors = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.718050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.718277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.718453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.718695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.718925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.719155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] standard_threads = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.719384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.719622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.719796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.719970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_journal = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.720143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_json = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.720314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_stderr = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.720488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_syslog = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.720720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.720970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.721231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.721490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.721726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.721972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.722206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.722448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.722681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.722923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.723168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.723416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.723648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.723890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.724122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.724365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.724605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.724836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.725084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.725372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.725614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.725848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.726097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.726354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.726608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.726853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.727102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.727332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.727558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.727790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.728032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.728275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.728502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.728729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.728971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.729236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.729473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.729717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.729944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.730177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.730406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.730632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.730859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.731084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.731366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.731545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.731784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.732009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.732240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.732466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.732698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.732927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.733187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.733417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.733650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.733881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.734112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.734342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.734575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.734799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.735040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.735271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.735504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.735732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.735986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.736221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.736456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.736689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.736922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.737297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.737297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.737774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.737957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.738264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.738500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.738733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.738977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.739213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.739445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.739686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.739921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.740154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.740385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.740617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.740848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.741106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.741367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.741605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.741838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.742076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.742308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.742540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.743293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.743521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.743748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.743955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 13:41:01.744035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.744152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 13:41:01.744231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.744341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] command line args: [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 13:41:01.744419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.744604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 13:41:01.744683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.744780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ================================================================================ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 13:41:01.744857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.745031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.745110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.745297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_dir = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.745376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.745485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.745573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.745723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_source = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.745823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.745902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] debug = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.746036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.746137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.746273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.746358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.external = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.746450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.746527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.746697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.746775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.746941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.747020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.747166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.747250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.password = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.747328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_config_append = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.747410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.token = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.747504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.747647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.747725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_dir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.747839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.747922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.748070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.748148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.748312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.748390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.748553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.748630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.748794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.748875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.748970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.749048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.749193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.749277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.749424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.749532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.749665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.749769 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.749880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.749982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.750109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.750214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.750344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.750449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.750574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.750685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.750813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.750897 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.751043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.751162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.751313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.751400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.751562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.751797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.91/identity {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.751880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.751966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] publish_errors = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.752070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.752191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.752328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.752447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.752566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.752653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.752807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.752887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.753034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.753122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.753293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] standard_threads = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.753418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.753511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.753595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.753748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.753830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.753954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.754078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.754176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_journal = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.754259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_json = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.754333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.754451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_stderr = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.754527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.754599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_syslog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.754739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.754834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:41:01.754972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.755926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.756939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.757908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.758063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.758141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.758296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.758405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759291 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.759873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.760852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.761965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.762973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.763931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.764863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.765940 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.766947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.767872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768769 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.768922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769246 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.769957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770414 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.770866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.771967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.772972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.773963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.774871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.775959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.776967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.777906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.778990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.779997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.external = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.password = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.780951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.token = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.781964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.782916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.783051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.783251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.783344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.783490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.783729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.783808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.783960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.784925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.785915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.786975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.787923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.788880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.789866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.790967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.791971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.792314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.792639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.792937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.793199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.793297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.793454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.793651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.793738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.793947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.794025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.794205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.794293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.794449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.794527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.794767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.794851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.795009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.795104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.795285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.795609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.795856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.796092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.796297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.796383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.796738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.796816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.797006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.797268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.797348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.797525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.797606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.797770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.797849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 13:41:01.798763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.798989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.799240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.799502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.799994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.800242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.800492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.800723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.800962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.801218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.801460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.801696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.801938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.802180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.802415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.802663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.802911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.803141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.803370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.803616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.803862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.804097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.804330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.804563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.804791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.805029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.805828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.806058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.806302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.806544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.806786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.807018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.807297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.807485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.807695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.807926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.808267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.808514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.808759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.809003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.809272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.809519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.809746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.809989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.810224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.810459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.810695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.810930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.811171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.811416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.811651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.811901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.812129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.812371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.812610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.812842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.813077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.813345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.813656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.813912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.814154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.814392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.814639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.814886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.815121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.815369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.815617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.815849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.816092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.816326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.816561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.816840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.817086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.817339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.817588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.817833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.818078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.818323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.818569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.818812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.819056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.819301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.819532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.819777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.820022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.820266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.820508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.821472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.821727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.821979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.822228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.822476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.822729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.822972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.823223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.823474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.823712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.823941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.824171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.824400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.824647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.824897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.825142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.825402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.825636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.825865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.826098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.826332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.826576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.826809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.827060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.827302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.827539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.827780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.828022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.828265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.828499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.828731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.828963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.829224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.829469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.829697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.829937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.830224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.830490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.830733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.830822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.830979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.831061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.831205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.831287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.831451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.831684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.831918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.832150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.832384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.832631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.832865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.833105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.833361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.833605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.833838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.834077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.834304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.834542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.834773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:41:01.834941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 13:41:01.839823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.860102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.860330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.860556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.868591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.868819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.871118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.890215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.890444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.921184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.928107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.972726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.972948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.976143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:01.976374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.038261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.038488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.038711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.038797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.038935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.039029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.039158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.039252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.039371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.039469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.040113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.040329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.063647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.063833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.064096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.064237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.064370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.064500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.064628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.064790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.064915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.065039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.065184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.065311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.065436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.065587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.076238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.076532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.088624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.088854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.089995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.090069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.090208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.090421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:41:02.108231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.108434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:41:02.108700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.108787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.108913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:41:02.109001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:41:02.109237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.109324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.109456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:41:02.109540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.109724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.109810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.109975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.110057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.110235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.110488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.122120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.122391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.122653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.122930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.123304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.123579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.123838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.124118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.131021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.132233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.134331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.134606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.134888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.135204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.135446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.135547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.135704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.135896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136159 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.136987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.137173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.137283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.137556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.137825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.138094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.138360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.163555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.164463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.165693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.165971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.166749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.167026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.171307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.171529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.171802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.172283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.172561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.172837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.179548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.179778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.180947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.181256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.184855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.185279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.185563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.185737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.185863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.186135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.186217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.186405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.186489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.186683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.186771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.186947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.187027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.187217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.187486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.187750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.205537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.205816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.206227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.206505 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.210483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.211116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.214152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.214444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.214742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.215037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.219468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.219980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.223056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.223763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.226798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.227604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.227895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.228045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.228181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.228340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.228452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.228633 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.228719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.228918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.229216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.229500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.248134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.248415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.248674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.248930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.249015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.249292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.249555 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.249814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.256914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.258008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.258658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.258932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.259726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.259997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.262021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.262293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.262564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.262837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.263069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.263160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.263346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.263621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.263898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.264173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.276481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.276756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.277036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.277336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.277610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.277880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.278002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.278166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.278289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.278493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.278778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.279055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.279332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.279614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.296539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.298210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.300092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.301751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.303414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.305019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.305122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.307240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.307371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.307522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.307799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.308997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.309310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.309600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.315165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.316778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.316980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.318606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.320015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.320310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.320591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.320901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.322062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.322293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.322574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.322857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.330003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.330358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.330796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.332006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.332290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.332668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.341245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.341524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.341808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.342081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.342351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.342622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.342893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.343174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.343440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.343720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.343987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.344243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.344504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.344777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:41:02.344862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.345269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.345744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.346214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.346690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.347169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.347662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.348120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.348600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.349072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.349577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.350036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.350524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.350989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:41:02.385269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5607872d49c0 pid: 80237 (default app) May 26 13:41:02.388490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dc426-e1c3-4cd9-b8a8-c219d22c18c2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:02.388794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dc426-e1c3-4cd9-b8a8-c219d22c18c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:02.389071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dc426-e1c3-4cd9-b8a8-c219d22c18c2 None None] PATH_INFO: `/v3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:02.393499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1/1] 10.222.0.91 () {50 vars in 791 bytes} [Thu May 26 13:41:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:02.413245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5607872d49c0 pid: 80236 (default app) May 26 13:41:10.364587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58448279-ff51-41dc-8237-0bf2c5013976 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:10.365144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58448279-ff51-41dc-8237-0bf2c5013976 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:10.365301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58448279-ff51-41dc-8237-0bf2c5013976 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:10.366653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 2/2] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:10.374201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:10.374629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:10.374762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:10.444056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: INFO dbcounter [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] Registered counter for database keystone May 26 13:41:10.466164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG oslo_db.sqlalchemy.engines [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80236) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 13:41:10.472584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writer thread running {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 13:41:10.659459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG passlib.handlers.bcrypt [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80236) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 13:41:10.659871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG passlib.handlers.bcrypt [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80236) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 13:41:10.716072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:10.717640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:10.844446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1/3] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 13:41:10.970354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58448279-ff51-41dc-8237-0bf2c5013976 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:11.040106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: INFO dbcounter [None req-58448279-ff51-41dc-8237-0bf2c5013976 None None] Registered counter for database keystone May 26 13:41:11.061371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG oslo_db.sqlalchemy.engines [None req-58448279-ff51-41dc-8237-0bf2c5013976 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80237) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 13:41:11.067886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writer thread running {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 13:41:11.190047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58448279-ff51-41dc-8237-0bf2c5013976 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:11.194445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fafb7-0014-4531-acbd-e0e09b2fea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:11.195123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fafb7-0014-4531-acbd-e0e09b2fea93 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:11.195405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fafb7-0014-4531-acbd-e0e09b2fea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:11.195731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fafb7-0014-4531-acbd-e0e09b2fea93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:11.204148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fafb7-0014-4531-acbd-e0e09b2fea93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:11.256833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6a7fafb7-0014-4531-acbd-e0e09b2fea93 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 13:41:11.257103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fafb7-0014-4531-acbd-e0e09b2fea93 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:11.267049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 3/4] 10.222.0.91 () {58 vars in 1324 bytes} [Thu May 26 13:41:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 13:41:11.273324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:11.287551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4b526-b95a-446a-b90d-b4f6ccbf3de1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:11.292517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d954e98-5ab5-411a-9b76-6e08c67234a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:11.293656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d954e98-5ab5-411a-9b76-6e08c67234a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:11.293943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d954e98-5ab5-411a-9b76-6e08c67234a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:11.294216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d954e98-5ab5-411a-9b76-6e08c67234a9 admin admin] PATH_INFO: `/v3/projects/e741295126a84576b8368706b7ffe76d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:11.295846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d954e98-5ab5-411a-9b76-6e08c67234a9 admin admin] RBAC: Authorizing `identity:get_project(project_id=e741295126a84576b8368706b7ffe76d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:11.454843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4d954e98-5ab5-411a-9b76-6e08c67234a9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 13:41:11.455502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d954e98-5ab5-411a-9b76-6e08c67234a9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:11.459879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 2/5] 10.222.0.91 () {56 vars in 1207 bytes} [Thu May 26 13:41:11 2022] GET /identity/v3/projects/e741295126a84576b8368706b7ffe76d => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:13.647932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f3fb3-d132-4ed9-87a7-f03ed358539b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:13.648426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f3fb3-d132-4ed9-87a7-f03ed358539b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:13.648426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f3fb3-d132-4ed9-87a7-f03ed358539b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:13.653023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 4/6] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:13.658698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b06d3-d3e2-4c7c-ab55-3ec2b466276e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:13.659359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b06d3-d3e2-4c7c-ab55-3ec2b466276e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:13.659359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b06d3-d3e2-4c7c-ab55-3ec2b466276e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:13.679245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1e4b06d3-d3e2-4c7c-ab55-3ec2b466276e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:13.680767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1e4b06d3-d3e2-4c7c-ab55-3ec2b466276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:13.688678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 3/7] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 13:41:13.710757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f3fb3-d132-4ed9-87a7-f03ed358539b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:13.723407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f3fb3-d132-4ed9-87a7-f03ed358539b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:13.728263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867351e8-1ef5-432f-a58f-a225dc55de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:13.728899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867351e8-1ef5-432f-a58f-a225dc55de71 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:13.729209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867351e8-1ef5-432f-a58f-a225dc55de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:13.729495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867351e8-1ef5-432f-a58f-a225dc55de71 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:13.737298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867351e8-1ef5-432f-a58f-a225dc55de71 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:13.739079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867351e8-1ef5-432f-a58f-a225dc55de71 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:13.747855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 5/8] 10.222.0.91 () {58 vars in 1324 bytes} [Thu May 26 13:41:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:41:13.753530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b06d3-d3e2-4c7c-ab55-3ec2b466276e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:13.763029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b06d3-d3e2-4c7c-ab55-3ec2b466276e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:13.767712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376c9dc-83b8-4e76-ab41-304bdf1e4c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:13.768562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c9dc-83b8-4e76-ab41-304bdf1e4c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:13.768853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c9dc-83b8-4e76-ab41-304bdf1e4c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:13.769127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c9dc-83b8-4e76-ab41-304bdf1e4c71 admin admin] PATH_INFO: `/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:13.770549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376c9dc-83b8-4e76-ab41-304bdf1e4c71 admin admin] RBAC: Authorizing `identity:get_user(user_id=42b8c6ab6fbb49deaec22b8b6964d6a5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:13.772386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376c9dc-83b8-4e76-ab41-304bdf1e4c71 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:13.774556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 4/9] 10.222.0.91 () {56 vars in 1198 bytes} [Thu May 26 13:41:13 2022] GET /identity/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:16.075059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88792a-7e35-4044-a6c7-8dbdb2b98e05 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.075613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88792a-7e35-4044-a6c7-8dbdb2b98e05 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.075734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88792a-7e35-4044-a6c7-8dbdb2b98e05 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.077105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 6/10] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:16.082584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ca51-6945-4d31-b4ff-63fb54cef511 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.083032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ca51-6945-4d31-b4ff-63fb54cef511 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.083132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869ca51-6945-4d31-b4ff-63fb54cef511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.099683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5869ca51-6945-4d31-b4ff-63fb54cef511 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:16.100853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5869ca51-6945-4d31-b4ff-63fb54cef511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:16.119366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b8e6-badc-4c1b-a62c-6963376cc05c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.119652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b8e6-badc-4c1b-a62c-6963376cc05c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.119926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b8e6-badc-4c1b-a62c-6963376cc05c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.121312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 7/11] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:16.126338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.126626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.126909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.167107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80237) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 13:41:16.167537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80237) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 13:41:16.184404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 5/12] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:16.209672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ca51-6945-4d31-b4ff-63fb54cef511 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.215277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:16.216881 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:16.240821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:16.318525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 8/13] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:16.325251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869ca51-6945-4d31-b4ff-63fb54cef511 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:16.345383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.359971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:16.360817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.361105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.361425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.374202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:16.385574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:16.386684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:16.387500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 6/14] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:16.392905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.455113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8f4c9-29f9-415f-b583-fa4f37aead3e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:16.457396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:16.481626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:16.496551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:16.497507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.497801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.498080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.500337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b10a06-cf71-4d68-a12f-c815ec7e67b0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:16.534577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:16.535255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.535538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.535813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.536577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:16.536883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:16.538331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:16.551451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 7/15] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:16.558761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.570704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:16.572415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:16.671515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821585dc-cdfe-495a-841d-128a335f21ae None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:16.675430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 9/16] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:41:16.681011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.684329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:16.707006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:16.707929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.708346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.708658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] PATH_INFO: `/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.710026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] RBAC: Authorizing `identity:get_user(user_id=42b8c6ab6fbb49deaec22b8b6964d6a5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:16.712127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:16.713916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 8/17] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:16.719580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.735089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:16.788225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6688b96-022e-4614-8b3a-d7eef644252c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:16.824773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:16.826309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.826309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.826495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.829595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27816e52-cbb6-40b0-adda-2be64084ba22 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:16.847892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:16.849573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:16.868131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:16.868928 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:16.869292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:16.869577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:16.870711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:41:16.871233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:16.872050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:16.872864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:41:16.872864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:41:16.872864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:41:16.872864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:16.873071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:16.873870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 9/18] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:16.877363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 10/19] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:16.878517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.879948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:16.938598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:16.964166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:16.984528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08dae25-c292-4c96-bb85-b1f9d2d61675 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:16.988274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f524-37f0-4cad-8aea-13bf44a557a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:17.025709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439b0db-fc71-4a63-bd5d-58fec652fd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:17.026474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439b0db-fc71-4a63-bd5d-58fec652fd2b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:17.026784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439b0db-fc71-4a63-bd5d-58fec652fd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:17.027056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23e3a3-26bf-4a42-a49e-07ec7a84120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:17.027298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439b0db-fc71-4a63-bd5d-58fec652fd2b None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:17.027929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439b0db-fc71-4a63-bd5d-58fec652fd2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:17.028100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23e3a3-26bf-4a42-a49e-07ec7a84120f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:17.028247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439b0db-fc71-4a63-bd5d-58fec652fd2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:17.028392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23e3a3-26bf-4a42-a49e-07ec7a84120f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:17.028769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23e3a3-26bf-4a42-a49e-07ec7a84120f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:17.029758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439b0db-fc71-4a63-bd5d-58fec652fd2b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:17.029916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23e3a3-26bf-4a42-a49e-07ec7a84120f None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.domain.id=default, user.id=42b8c6ab6fbb49deaec22b8b6964d6a5) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:17.030249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23e3a3-26bf-4a42-a49e-07ec7a84120f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:17.031928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23e3a3-26bf-4a42-a49e-07ec7a84120f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:17.041388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 11/20] 10.222.0.91 () {56 vars in 1322 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/role_assignments?user.id=42b8c6ab6fbb49deaec22b8b6964d6a5&scope.domain.id=default&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:17.048538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 10/21] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:19.762929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13747f1-9ddb-4c6e-8d43-6e38f4bf62a1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:19.764056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13747f1-9ddb-4c6e-8d43-6e38f4bf62a1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:19.764537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13747f1-9ddb-4c6e-8d43-6e38f4bf62a1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:19.766177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 12/22] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:19.771022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f27f53-26f6-446f-9271-71fca92c3bf2 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:19.771909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f27f53-26f6-446f-9271-71fca92c3bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:19.772403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f27f53-26f6-446f-9271-71fca92c3bf2 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:19.774750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 11/23] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:19.776888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e268b-7553-4321-9b8b-a2be9898bf56 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:19.777566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e268b-7553-4321-9b8b-a2be9898bf56 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:19.778189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e268b-7553-4321-9b8b-a2be9898bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:19.782901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c20129-93af-4b41-abe9-ad8792abbb22 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:19.783526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c20129-93af-4b41-abe9-ad8792abbb22 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:19.784127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c20129-93af-4b41-abe9-ad8792abbb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:19.823906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-09c20129-93af-4b41-abe9-ad8792abbb22 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:19.830344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1c8e268b-7553-4321-9b8b-a2be9898bf56 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:19.831364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-09c20129-93af-4b41-abe9-ad8792abbb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:19.831698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1c8e268b-7553-4321-9b8b-a2be9898bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:20.095750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 12/24] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:20.138240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c20129-93af-4b41-abe9-ad8792abbb22 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:20.171702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 13/25] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:20.182230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31846df-570d-40bf-86a9-d84838dd612e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.182516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31846df-570d-40bf-86a9-d84838dd612e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.182958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31846df-570d-40bf-86a9-d84838dd612e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.184517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 14/26] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:20.196102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41845c-a19c-4661-8837-03e5b233b898 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.196102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41845c-a19c-4661-8837-03e5b233b898 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.196102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41845c-a19c-4661-8837-03e5b233b898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.238228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2b41845c-a19c-4661-8837-03e5b233b898 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:20.242897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b41845c-a19c-4661-8837-03e5b233b898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:20.305542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:20.365638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:20.371660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 15/27] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:20.378418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41845c-a19c-4661-8837-03e5b233b898 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:20.400663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c20129-93af-4b41-abe9-ad8792abbb22 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:20.451802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb2d5a-2f7d-40b3-b0cc-d345207bf24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:20.453477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2d5a-2f7d-40b3-b0cc-d345207bf24d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.454241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2d5a-2f7d-40b3-b0cc-d345207bf24d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.454704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2d5a-2f7d-40b3-b0cc-d345207bf24d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.468907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2d5a-2f7d-40b3-b0cc-d345207bf24d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:20.471548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2d5a-2f7d-40b3-b0cc-d345207bf24d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4dfb2d5a-2f7d-40b3-b0cc-d345207bf24d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:41:20.499490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:20.500842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:20.502411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:41:20.502411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:41:20.502411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:41:20.502411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:20.502788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 13/28] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:20.504391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f01420-620e-4740-b2ca-1109d0cdc4b8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.504391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f01420-620e-4740-b2ca-1109d0cdc4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.510280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f01420-620e-4740-b2ca-1109d0cdc4b8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.510280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 14/29] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:20.510280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f01420-620e-4740-b2ca-1109d0cdc4b8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:20.530504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41845c-a19c-4661-8837-03e5b233b898 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:20.580882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03347e0-3294-4a0d-8f68-569567c71c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:20.582549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03347e0-3294-4a0d-8f68-569567c71c10 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.583036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03347e0-3294-4a0d-8f68-569567c71c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.583631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03347e0-3294-4a0d-8f68-569567c71c10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.601421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03347e0-3294-4a0d-8f68-569567c71c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e03347e0-3294-4a0d-8f68-569567c71c10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:20.611394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:20.612456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:20.613655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 16/30] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:20.614544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600e9e-f1e8-430c-972a-791ad71ae290 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.614637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600e9e-f1e8-430c-972a-791ad71ae290 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.614885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600e9e-f1e8-430c-972a-791ad71ae290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.634012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-88600e9e-f1e8-430c-972a-791ad71ae290 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:20.635116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-88600e9e-f1e8-430c-972a-791ad71ae290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:20.655020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f01420-620e-4740-b2ca-1109d0cdc4b8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:20.682234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:20.700282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:20.722191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 17/31] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:20.724099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:20.725333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.725821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.726387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.727957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:20.730002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88600e9e-f1e8-430c-972a-791ad71ae290 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:20.730554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:20.857363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88600e9e-f1e8-430c-972a-791ad71ae290 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:20.884821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 15/32] 10.222.0.91 () {60 vars in 1131 bytes} [Thu May 26 13:41:20 2022] POST /identity/v3/roles => generated 219 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:20.886632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:20.900928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:20.901649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:20.901932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:20.902209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:20.902952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:20.903257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:20.905310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:20.931115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 18/33] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:20.932375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:20.981463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:21.028974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1f838-b0fa-41c3-b0a4-d82aff239db4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:21.071047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:21.072066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.072534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.073088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.074298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:21.074770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:21.076677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:21.080570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3821e39-1f1c-41f2-bcb7-9f74305eebcf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:21.087816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 16/34] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:21.088926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:21.133827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:21.134947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.135186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.135459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.136552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:21.138009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:21.156827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:21.244219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bda315-e944-4688-9639-aa6676aebe14 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:21.288347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd29d5f-750e-4cc0-bd21-387804366ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:21.289378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd29d5f-750e-4cc0-bd21-387804366ef3 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.289916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd29d5f-750e-4cc0-bd21-387804366ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.290345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd29d5f-750e-4cc0-bd21-387804366ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.291218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd29d5f-750e-4cc0-bd21-387804366ef3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:21.292963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd29d5f-750e-4cc0-bd21-387804366ef3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:21.295117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 19/35] 10.222.0.91 () {60 vars in 1131 bytes} [Thu May 26 13:41:20 2022] POST /identity/v3/roles => generated 225 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:21.296284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:21.353777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 17/36] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:41:21 2022] POST /identity/v3/projects => generated 301 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:21.355640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594315-4e66-4a05-beb4-a82dc14294c1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.355937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594315-4e66-4a05-beb4-a82dc14294c1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.356219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594315-4e66-4a05-beb4-a82dc14294c1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.361920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 18/37] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:21.368705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e7d6-f092-400d-98f2-89698f92026e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.369014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e7d6-f092-400d-98f2-89698f92026e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.369305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e7d6-f092-400d-98f2-89698f92026e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.408761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-06b0e7d6-f092-400d-98f2-89698f92026e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:21.410496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-06b0e7d6-f092-400d-98f2-89698f92026e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:21.469553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:21.487940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240d9c-9f9f-4cff-9198-ecbe76cd23c4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:21.513651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 19/38] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:21.515622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ccec3-37e2-4959-99ad-6b812c6046c3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.515924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ccec3-37e2-4959-99ad-6b812c6046c3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.516210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ccec3-37e2-4959-99ad-6b812c6046c3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.517997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 20/39] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:21.524860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125cda-068f-4c85-b61c-e17db7343e73 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.525201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125cda-068f-4c85-b61c-e17db7343e73 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.525474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125cda-068f-4c85-b61c-e17db7343e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.548677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-38125cda-068f-4c85-b61c-e17db7343e73 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:21.551245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-38125cda-068f-4c85-b61c-e17db7343e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:21.572140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:21.573103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.573923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.574220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] PATH_INFO: `/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.576263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] RBAC: Authorizing `identity:get_user(user_id=42b8c6ab6fbb49deaec22b8b6964d6a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:21.578072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:21.579832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 20/40] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:21 2022] GET /identity/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5 => generated 247 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:21.580708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:21.584180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:21.654894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 21/41] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:21.655797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38125cda-068f-4c85-b61c-e17db7343e73 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:21.718993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27e301-2cfc-4437-8b0d-f4ebc0f4b855 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:21.755116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:21.755966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.756208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.756487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.757848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:21.759307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:21.760962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 21/42] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:21.761727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:21.784666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38125cda-068f-4c85-b61c-e17db7343e73 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:21.789477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:21.820327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:21.821229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.821438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.821722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.822913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:21.824383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:21.830543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 22/43] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:21.830543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:21.857632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:21.866869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b5574-d3eb-4de1-b7df-7777f7a27262 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:21.902883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:21.903563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:21.903853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:21.904130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:21.904902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:21.906340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:21.923736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 22/44] 10.222.0.91 () {60 vars in 1131 bytes} [Thu May 26 13:41:21 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:21.924836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:21.958317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902787b-d9b8-4380-b6ed-42cb9fdca548 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:22.000684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46c87d-13ef-4c70-936a-4067c8cc0e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:22.001321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46c87d-13ef-4c70-936a-4067c8cc0e10 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:22.001627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46c87d-13ef-4c70-936a-4067c8cc0e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:22.001907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46c87d-13ef-4c70-936a-4067c8cc0e10 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:22.002646 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46c87d-13ef-4c70-936a-4067c8cc0e10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:22.004225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46c87d-13ef-4c70-936a-4067c8cc0e10 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:22.033330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 23/45] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:41:21 2022] POST /identity/v3/projects => generated 312 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:22.055241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:22.078618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac54d3-02da-450a-babd-3ab2f720a2b3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:22.139792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068bd4f-1799-42da-87a2-eab1814bdf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:22.141016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068bd4f-1799-42da-87a2-eab1814bdf9f None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:22.141016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068bd4f-1799-42da-87a2-eab1814bdf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:22.141317 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068bd4f-1799-42da-87a2-eab1814bdf9f None admin] PATH_INFO: `/v3/domains/default/users/42b8c6ab6fbb49deaec22b8b6964d6a5/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:22.143526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068bd4f-1799-42da-87a2-eab1814bdf9f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=42b8c6ab6fbb49deaec22b8b6964d6a5, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:22.145025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068bd4f-1799-42da-87a2-eab1814bdf9f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:22.163752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 23/46] 10.222.0.91 () {58 vars in 1361 bytes} [Thu May 26 13:41:21 2022] PUT /identity/v3/domains/default/users/42b8c6ab6fbb49deaec22b8b6964d6a5/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:24.300038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d5cc6-fe81-4dfe-88de-556c88c1f3c9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.300554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d5cc6-fe81-4dfe-88de-556c88c1f3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.300664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d5cc6-fe81-4dfe-88de-556c88c1f3c9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.302025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 24/47] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:24.307417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fd3bd-b0e3-4b89-b64a-d99851f6eadf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.307853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fd3bd-b0e3-4b89-b64a-d99851f6eadf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.307990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fd3bd-b0e3-4b89-b64a-d99851f6eadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.327446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-488fd3bd-b0e3-4b89-b64a-d99851f6eadf None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:24.328982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-488fd3bd-b0e3-4b89-b64a-d99851f6eadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:24.332012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0e737-2271-4446-87d7-46cbda47ab65 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.332305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0e737-2271-4446-87d7-46cbda47ab65 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.332588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0e737-2271-4446-87d7-46cbda47ab65 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.334157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 25/48] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:24.340213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf83899-ba2c-4950-9e4a-59ce92c8ddac None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.340509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf83899-ba2c-4950-9e4a-59ce92c8ddac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.340795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf83899-ba2c-4950-9e4a-59ce92c8ddac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.358003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-edf83899-ba2c-4950-9e4a-59ce92c8ddac None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:24.359249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-edf83899-ba2c-4950-9e4a-59ce92c8ddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:24.400253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 24/49] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:24.423509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fd3bd-b0e3-4b89-b64a-d99851f6eadf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:24.433909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 26/50] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:24.456359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf83899-ba2c-4950-9e4a-59ce92c8ddac None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:24.464653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:24.504624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:24.530819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fd3bd-b0e3-4b89-b64a-d99851f6eadf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:24.562486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf83899-ba2c-4950-9e4a-59ce92c8ddac None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:24.566814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:24.567615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.567894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.568167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.569383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:24.571540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:24.573203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 25/51] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:24.578896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:24.599643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:24.600457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.600748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.601032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.611464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:24.619433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:24.620324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:24.621117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 27/52] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:24.626445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:24.683403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f47f-2360-417a-b165-40cf75685360 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:24.693603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:24.720580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:24.721146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.721455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.721736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.722477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:24.723935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:24.730521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9e2b-88a2-48b6-a370-9a19ba1a6c43 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:24.739191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:24.752289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 26/53] 10.222.0.91 () {60 vars in 1140 bytes} [Thu May 26 13:41:24 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:41:24.768097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:24.768807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.769097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.769408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.770196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:24.770509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:24.772011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:24.782112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 28/54] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:24.788034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:24.903991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050c620-ace9-4497-87a4-0063383ec5f5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:24.944136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:24.945073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:24.945880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:24.946157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] PATH_INFO: `/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:24.947374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] RBAC: Authorizing `identity:get_user(user_id=42b8c6ab6fbb49deaec22b8b6964d6a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:24.948840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:24.950714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 27/55] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:24 2022] GET /identity/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:24.955946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:25.063049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46fdd6-1718-4acd-841c-45270998cb66 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:25.090077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:25.098936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941c25b-c62c-465c-8355-242527426ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:25.099733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941c25b-c62c-465c-8355-242527426ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:25.100019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941c25b-c62c-465c-8355-242527426ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:25.100298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941c25b-c62c-465c-8355-242527426ed2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:25.101511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941c25b-c62c-465c-8355-242527426ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:25.103037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941c25b-c62c-465c-8355-242527426ed2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:25.104874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 29/56] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:25.110411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:25.174533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:25.223041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe943e-0c38-44db-b473-24d6ed11daf5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:25.264820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e897b-31ae-4074-84d9-71c24042ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:25.265510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e897b-31ae-4074-84d9-71c24042ac85 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:25.265798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e897b-31ae-4074-84d9-71c24042ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:25.266074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e897b-31ae-4074-84d9-71c24042ac85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:25.266928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e897b-31ae-4074-84d9-71c24042ac85 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.domain.id=default, user.id=42b8c6ab6fbb49deaec22b8b6964d6a5) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:25.267236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e897b-31ae-4074-84d9-71c24042ac85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:25.268701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e897b-31ae-4074-84d9-71c24042ac85 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:25.275072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 28/57] 10.222.0.91 () {56 vars in 1322 bytes} [Thu May 26 13:41:25 2022] GET /identity/v3/role_assignments?user.id=42b8c6ab6fbb49deaec22b8b6964d6a5&scope.domain.id=default&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 530 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:27.000764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9510a18-d1f7-4b67-a886-24e97f56d1a0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:27.001337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9510a18-d1f7-4b67-a886-24e97f56d1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:27.001337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9510a18-d1f7-4b67-a886-24e97f56d1a0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:27.002745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 30/58] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:27.008120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a97711-dd5f-401a-a77b-f906e8204076 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:27.008557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a97711-dd5f-401a-a77b-f906e8204076 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:27.008666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a97711-dd5f-401a-a77b-f906e8204076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:27.025461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-85a97711-dd5f-401a-a77b-f906e8204076 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:27.026656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85a97711-dd5f-401a-a77b-f906e8204076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:27.093338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 29/59] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:27.118097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9510a18-d1f7-4b67-a886-24e97f56d1a0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:27.220761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9510a18-d1f7-4b67-a886-24e97f56d1a0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:27.254366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083cf4f-50b8-4bd1-b63b-053d50863ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:27.255043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cf4f-50b8-4bd1-b63b-053d50863ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:27.255313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cf4f-50b8-4bd1-b63b-053d50863ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:27.255592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cf4f-50b8-4bd1-b63b-053d50863ef7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:27.256771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083cf4f-50b8-4bd1-b63b-053d50863ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:27.258267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083cf4f-50b8-4bd1-b63b-053d50863ef7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:27.259885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 31/60] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:27.265424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a97711-dd5f-401a-a77b-f906e8204076 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:27.323747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:27.365745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a97711-dd5f-401a-a77b-f906e8204076 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:27.398658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d58fc-58a8-4d61-a7e0-d971770c3360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:27.399331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d58fc-58a8-4d61-a7e0-d971770c3360 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:27.399616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d58fc-58a8-4d61-a7e0-d971770c3360 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:27.399890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d58fc-58a8-4d61-a7e0-d971770c3360 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:27.400671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d58fc-58a8-4d61-a7e0-d971770c3360 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:27.402147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d58fc-58a8-4d61-a7e0-d971770c3360 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:27.427298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 30/61] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:41:27 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:31.322599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751fd15-2a0d-4968-a832-22380ef7dceb None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.323628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751fd15-2a0d-4968-a832-22380ef7dceb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.324164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751fd15-2a0d-4968-a832-22380ef7dceb None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.325867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 32/62] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:31.331771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca78b2-6f40-498e-bcb4-41450c07f942 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.332734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca78b2-6f40-498e-bcb4-41450c07f942 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.333251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca78b2-6f40-498e-bcb4-41450c07f942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.354161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-27ca78b2-6f40-498e-bcb4-41450c07f942 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:31.355692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-27ca78b2-6f40-498e-bcb4-41450c07f942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:31.386706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b02ce9-31b9-46a6-a528-874fd25826e0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.387313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b02ce9-31b9-46a6-a528-874fd25826e0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.387786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b02ce9-31b9-46a6-a528-874fd25826e0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.389418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 33/63] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:31.401506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e4a59-9865-41b1-ae7e-10f7c7e1dcaa None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.402111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e4a59-9865-41b1-ae7e-10f7c7e1dcaa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.402591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e4a59-9865-41b1-ae7e-10f7c7e1dcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.433032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-391e4a59-9865-41b1-ae7e-10f7c7e1dcaa None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:31.434698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-391e4a59-9865-41b1-ae7e-10f7c7e1dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:31.448448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 31/64] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:31.467088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca78b2-6f40-498e-bcb4-41450c07f942 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:31.524673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:31.530037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:31.532924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 34/65] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:31.534494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a1f68-47e0-4f2b-87e7-74e3898fc6a6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.534789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a1f68-47e0-4f2b-87e7-74e3898fc6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.535073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a1f68-47e0-4f2b-87e7-74e3898fc6a6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.536352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 35/66] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:31.537453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152190b-9a52-407e-9834-7576e7213ff7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.537889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152190b-9a52-407e-9834-7576e7213ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.538350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152190b-9a52-407e-9834-7576e7213ff7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.539815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 36/67] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:31.546955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05668876-3214-4ae4-acfd-c895fa2f7d1a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.547257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05668876-3214-4ae4-acfd-c895fa2f7d1a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.547550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05668876-3214-4ae4-acfd-c895fa2f7d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.567470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-05668876-3214-4ae4-acfd-c895fa2f7d1a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:31.568665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-05668876-3214-4ae4-acfd-c895fa2f7d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:31.604629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca78b2-6f40-498e-bcb4-41450c07f942 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:31.641944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 37/68] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:31.653386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b7a5-fe3b-49f8-a1d7-2bcfe6d70cb8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.653386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b7a5-fe3b-49f8-a1d7-2bcfe6d70cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.653386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b7a5-fe3b-49f8-a1d7-2bcfe6d70cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.654677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:31.655607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.655901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.656183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.669452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:31.671058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e6f2b7a5-fe3b-49f8-a1d7-2bcfe6d70cb8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:31.672758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e6f2b7a5-fe3b-49f8-a1d7-2bcfe6d70cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:31.678050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:31.678938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 32/69] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:31.680816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:31.749013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 38/70] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:31.750217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2b7a5-fe3b-49f8-a1d7-2bcfe6d70cb8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:31.800710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f772e-c805-4523-9f34-f4fc90163a94 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:31.803336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:31.816051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:31.840524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:31.841361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.841653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.841936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.852672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:31.859659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2b7a5-fe3b-49f8-a1d7-2bcfe6d70cb8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:31.860976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:31.862052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 33/71] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:31.863016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:31.897507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:31.898328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:31.898616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:31.898893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:31.900065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:31.901544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:31.903146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 39/72] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:31.903905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:31.968125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f38b1-8176-4c74-9848-e77639af87f3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.001928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c862e06-9265-471e-92f3-6f43ace731b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.002522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c862e06-9265-471e-92f3-6f43ace731b5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.002772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910e6d2-c6ff-4a0b-949b-63522c71ae38 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.002867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c862e06-9265-471e-92f3-6f43ace731b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.003071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c862e06-9265-471e-92f3-6f43ace731b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.003807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c862e06-9265-471e-92f3-6f43ace731b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:32.004121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c862e06-9265-471e-92f3-6f43ace731b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.005540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c862e06-9265-471e-92f3-6f43ace731b5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.017456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 34/73] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:32.018525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe65d-e26c-4879-9be6-0c6bb82998bd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.018812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe65d-e26c-4879-9be6-0c6bb82998bd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.019088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe65d-e26c-4879-9be6-0c6bb82998bd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.020331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 35/74] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:32.021018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe65d-e26c-4879-9be6-0c6bb82998bd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.029767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.042366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.043186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.043473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.043758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.047754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.055605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:32.063630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:32.064514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:32.065310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 40/75] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:41:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:32.065994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.127845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe65d-e26c-4879-9be6-0c6bb82998bd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.162007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.162705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333454-5464-47ae-b6a2-e43e689016e9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.162841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.163164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.163996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.164844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:32.165263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.166665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.179944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 36/76] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:32.179944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.202232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54dffe3-3b54-4392-b6f6-cc7f28a611e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.202925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54dffe3-3b54-4392-b6f6-cc7f28a611e6 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.203216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54dffe3-3b54-4392-b6f6-cc7f28a611e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.203497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54dffe3-3b54-4392-b6f6-cc7f28a611e6 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.204223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54dffe3-3b54-4392-b6f6-cc7f28a611e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.205738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54dffe3-3b54-4392-b6f6-cc7f28a611e6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.232527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 41/77] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:41:32 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:32.233670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da9a46-df72-49f2-8d9a-7eedfc23d129 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.233972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da9a46-df72-49f2-8d9a-7eedfc23d129 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.234259 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da9a46-df72-49f2-8d9a-7eedfc23d129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.252957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-88da9a46-df72-49f2-8d9a-7eedfc23d129 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:32.254156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-88da9a46-df72-49f2-8d9a-7eedfc23d129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:32.293186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.297292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.310436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470f092-625d-4b59-a8f7-4c4010df2bfa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.340412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 42/78] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:32.341997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da9a46-df72-49f2-8d9a-7eedfc23d129 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.358380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.360044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.360524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.361008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] PATH_INFO: `/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.362578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] RBAC: Authorizing `identity:get_user(user_id=42b8c6ab6fbb49deaec22b8b6964d6a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.364368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.366421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 37/79] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:32.367421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.463926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da9a46-df72-49f2-8d9a-7eedfc23d129 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.484344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf549a-2654-4a40-a89c-fc8a280a3064 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.506439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.507060 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.507391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.507678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.508455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:32.510107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.510287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.520912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 43/80] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:32.521860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.530205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.531130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.531413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.531691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.557775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.559518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.561079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 38/81] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:32.562632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.563917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.587569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.653811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd37c9-0efb-49f8-a5e4-341d96971e8a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.699189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed387474-c984-403b-a65a-74f2c818da93 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.700003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.700741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.701372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.701734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.744446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.745532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.745965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.746556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.761651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:32.770308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:32.771363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:32.772279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:32.774930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 39/82] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:32.775920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.791914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.793720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.834082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.869180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:32.889930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa562a6-a787-4f78-b96f-8e1f41a7b5f1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:32.927415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:32.928323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:32.928608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:32.928882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:32.929370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 44/83] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:32.929994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:32.930296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:32.931760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:32.933748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 40/84] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:32.934404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.032073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0d4ab-f1c0-4c2c-acdb-3899df1c910d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.042520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe084a44-88fb-43ca-a19f-b35737b5cf32 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.061445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.068677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.069481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.070377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.070725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.080240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.080917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.081227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.081511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.082374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:33.082683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:33.084181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:33.093669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 41/85] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:33.098966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.102948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.145793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:33.147485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:33.208650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04371dd5-a5f6-4fb7-aed7-9ed2fdc8aed4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.248004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.248829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.249386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.249499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.250720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:33.252281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:33.254135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 42/86] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:33 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:33.254924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.260551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 45/87] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:33.261721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.293140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.330321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.369090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c0af8-46be-458f-b7a4-9a9ba2aad381 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.371323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ad44-81d0-487c-aece-90109c1d23fb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.410743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.411872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.412303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.412725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.414137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:33.414425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.415141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.415497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.415842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.416485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:33.418153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 46/88] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:33 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:33.418918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.511460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:33.513186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:33.536269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.542458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe87cfa-19a7-4d9f-b412-c98d207c68b8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.589079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.589702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.589989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.590268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.591221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=42b8c6ab6fbb49deaec22b8b6964d6a5) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:33.591534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:33.592986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:33.598364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.602457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 47/89] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:33 2022] GET /identity/v3/role_assignments?user.id=42b8c6ab6fbb49deaec22b8b6964d6a5&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:33.603113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.659595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 43/90] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:33.660547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.805887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af622e8-c0f9-4b96-926b-e7e831868145 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.844898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dae819-2910-4a66-9823-a583dce1f81f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:33.855073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.856056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.856349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.856631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.857859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:33.859350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:33.861253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 48/91] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:33 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:33.861892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:33.900636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.907720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:33.908390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:33.908680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:33.908959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:33.983893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:33.985180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dacceb-e348-4759-a13a-a990389a3519 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:34.025103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:34.026023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:34.026951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.027389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.027683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:34.028898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:34.030406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:34.032505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 49/92] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:33 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:34.033466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:34.034812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:34.140786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:34.144809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec683be-e975-47b9-82c4-42a1ab50ae3c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:34.195833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:34.196515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.196867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.197174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:34.198104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=d6b97d3592ef443885d4636f8757e02c) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:34.198486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:34.205189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:34.206134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 50/93] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:34 2022] GET /identity/v3/role_assignments?user.id=d6b97d3592ef443885d4636f8757e02c&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:34.207024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:34.255796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 44/94] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:34.270295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:34.431762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c7e87-89fc-49c0-9bd7-2431354d8013 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:34.462927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:34.508802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:34.509645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.510113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.510548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:34.511695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] RBAC: Adding query filter params (role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=d6b97d3592ef443885d4636f8757e02c) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:34.512168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:34.514276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:34.524031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389637d-1284-4203-b1e9-bae99dccbfa8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:34.526833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 51/95] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:34 2022] GET /identity/v3/role_assignments?user.id=d6b97d3592ef443885d4636f8757e02c&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=33a3e99f754a4bc69e665330de985b76 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:34.595317 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db4a5d-7155-450f-9c92-097ed61c3ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:34.596510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db4a5d-7155-450f-9c92-097ed61c3ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.596948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db4a5d-7155-450f-9c92-097ed61c3ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.597396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db4a5d-7155-450f-9c92-097ed61c3ea7 None admin] PATH_INFO: `/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:34.602181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db4a5d-7155-450f-9c92-097ed61c3ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=9c35dc6046ec4f808d7e567c8eaaac5a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:34.603896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db4a5d-7155-450f-9c92-097ed61c3ea7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:34.605900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 45/96] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:34 2022] GET /identity/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a => generated 312 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:34.612273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:34.657342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:34.765927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48394d67-4f56-473a-be91-2a04e5af0247 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:34.812624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303923c4-4ea7-436e-8096-3dc601a38698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:34.813710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303923c4-4ea7-436e-8096-3dc601a38698 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.814221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303923c4-4ea7-436e-8096-3dc601a38698 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.814705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303923c4-4ea7-436e-8096-3dc601a38698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:34.815796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303923c4-4ea7-436e-8096-3dc601a38698 None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=9c35dc6046ec4f808d7e567c8eaaac5a, user.id=d6b97d3592ef443885d4636f8757e02c) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:34.816333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303923c4-4ea7-436e-8096-3dc601a38698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:34.818099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303923c4-4ea7-436e-8096-3dc601a38698 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:34.829992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 52/97] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:34 2022] GET /identity/v3/role_assignments?user.id=d6b97d3592ef443885d4636f8757e02c&scope.project.id=9c35dc6046ec4f808d7e567c8eaaac5a&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:34.977563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f598d1-96b5-4894-a318-0df441c3a9a0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.978099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f598d1-96b5-4894-a318-0df441c3a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.978489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999db7c-398b-4e92-bbf0-fcab8404cc64 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.978790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999db7c-398b-4e92-bbf0-fcab8404cc64 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.979028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f598d1-96b5-4894-a318-0df441c3a9a0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:34.979131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999db7c-398b-4e92-bbf0-fcab8404cc64 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:34.980449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 53/98] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:34.980893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 46/99] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:34.985671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5bedb-abae-4e72-a12f-be2480cb59f7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:34.985967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5bedb-abae-4e72-a12f-be2480cb59f7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:34.986255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5bedb-abae-4e72-a12f-be2480cb59f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:35.002867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98b16e-c7bd-45cc-8b6a-240a59577747 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:35.003184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98b16e-c7bd-45cc-8b6a-240a59577747 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:35.003426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98b16e-c7bd-45cc-8b6a-240a59577747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:35.014114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-04e5bedb-abae-4e72-a12f-be2480cb59f7 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:35.015326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-04e5bedb-abae-4e72-a12f-be2480cb59f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:35.039742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ee98b16e-c7bd-45cc-8b6a-240a59577747 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:35.044910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ee98b16e-c7bd-45cc-8b6a-240a59577747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:35.157046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 54/100] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:35.181833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 47/101] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:35.214740 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5bedb-abae-4e72-a12f-be2480cb59f7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:35.229648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98b16e-c7bd-45cc-8b6a-240a59577747 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:35.248637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:35.385257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:35.424691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5bedb-abae-4e72-a12f-be2480cb59f7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:35.469176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98b16e-c7bd-45cc-8b6a-240a59577747 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:35.516633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:35.518168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:35.518662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:35.519146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:35.521951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:35.523544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:35.525670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 55/102] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:35.531819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:35.575249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:35.576065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:35.576354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:35.576633 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:35.607701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:35.621176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:35.622142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:35.622953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 48/103] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:35.629402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:35.641659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:35.663850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703dca8-0228-43a9-9a40-ec7a7842c798 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:35.737838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:35.738512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:35.738806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:35.739077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:35.739808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:35.741285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:35.777522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed8146-1f90-4389-95ba-ac388bcde065 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:35.813625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 56/104] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:41:35 2022] POST /identity/v3/projects => generated 302 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:35.835090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:35.841945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:35.842628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:35.842921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:35.843200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:35.843960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:35.844271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:35.845767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:35.864672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 49/105] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:35.870980 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:35.993036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96620ef-d5aa-4f46-9c38-ce323777992c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:36.031557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:36.032463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:36.032752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:36.033033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] PATH_INFO: `/v3/users/52e2cacded774b22965a2814a91a511e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:36.045021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:36.049876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=52e2cacded774b22965a2814a91a511e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:36.051650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:36.057348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 57/106] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:35 2022] GET /identity/v3/users/52e2cacded774b22965a2814a91a511e => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:36.059959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:36.171141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd43c7-a453-4aa3-bfa8-d7f380542454 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:36.208710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:36.209419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:36.209710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:36.209989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:36.277611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:36.286102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:36.287671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:36.390109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 50/107] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:36.395514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:36.503642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c96e8-d61a-47ad-bf65-b1a5a8715ab7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:36.545916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2e2d2-7803-43fb-9bfe-c53034d4dd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:36.546800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2e2d2-7803-43fb-9bfe-c53034d4dd53 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:36.547156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2e2d2-7803-43fb-9bfe-c53034d4dd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:36.547640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2e2d2-7803-43fb-9bfe-c53034d4dd53 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:36.549364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2e2d2-7803-43fb-9bfe-c53034d4dd53 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:36.550840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2e2d2-7803-43fb-9bfe-c53034d4dd53 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:36.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 58/108] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:36 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:36.558365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:36.671641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335139d-7882-41b2-89d8-fe113255a89f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:36.677517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:36.714776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931c5d1-f13d-426b-9463-e44ed804dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:36.715505 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c5d1-f13d-426b-9463-e44ed804dc4e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:36.715793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c5d1-f13d-426b-9463-e44ed804dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:36.716119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c5d1-f13d-426b-9463-e44ed804dc4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:36.717030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931c5d1-f13d-426b-9463-e44ed804dc4e None admin] RBAC: Adding query filter params (role.id=3208335c205449e6998150af5b78eaa2, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=52e2cacded774b22965a2814a91a511e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:36.717381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931c5d1-f13d-426b-9463-e44ed804dc4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:36.718831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931c5d1-f13d-426b-9463-e44ed804dc4e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:36.725097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 51/109] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:36 2022] GET /identity/v3/role_assignments?user.id=52e2cacded774b22965a2814a91a511e&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=3208335c205449e6998150af5b78eaa2 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:37.143085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1772a4-40ce-4556-aad6-8a9fcfccc21a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:37.143471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1772a4-40ce-4556-aad6-8a9fcfccc21a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:37.143661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1772a4-40ce-4556-aad6-8a9fcfccc21a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:37.145186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 59/110] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:37.168303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6bc64-c224-47a2-a541-7259def13ca2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:37.168606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6bc64-c224-47a2-a541-7259def13ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:37.168875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6bc64-c224-47a2-a541-7259def13ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:37.191943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0bd6bc64-c224-47a2-a541-7259def13ca2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:37.191943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0bd6bc64-c224-47a2-a541-7259def13ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:37.280704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 52/111] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:37.315196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1772a4-40ce-4556-aad6-8a9fcfccc21a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:37.407839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:37.452722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1772a4-40ce-4556-aad6-8a9fcfccc21a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:37.494648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c8e8-5032-4584-9c7a-bf576371adce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:37.495460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c8e8-5032-4584-9c7a-bf576371adce None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:37.495743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c8e8-5032-4584-9c7a-bf576371adce None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:37.496019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c8e8-5032-4584-9c7a-bf576371adce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:37.513939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6c8e8-5032-4584-9c7a-bf576371adce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-34a6c8e8-5032-4584-9c7a-bf576371adce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:37.521989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:37.523487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:37.524766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 60/112] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:37.530440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6bc64-c224-47a2-a541-7259def13ca2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:37.582730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a102111-f748-4ade-b00b-190c1d15423d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:37.583237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a102111-f748-4ade-b00b-190c1d15423d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:37.583332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a102111-f748-4ade-b00b-190c1d15423d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:37.584727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 61/113] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:37.590996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f7b2c-8cee-4da6-be88-5dbc4de0d066 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:37.591282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f7b2c-8cee-4da6-be88-5dbc4de0d066 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:37.591567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f7b2c-8cee-4da6-be88-5dbc4de0d066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:37.610087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-921f7b2c-8cee-4da6-be88-5dbc4de0d066 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:37.611311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-921f7b2c-8cee-4da6-be88-5dbc4de0d066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:37.691929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 62/114] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:37.717243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f7b2c-8cee-4da6-be88-5dbc4de0d066 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:37.748542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:37.767519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6bc64-c224-47a2-a541-7259def13ca2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:37.807365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:37.808368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:37.808809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:37.809389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:37.810405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:37.810978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:37.812520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:37.822273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 53/115] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:37.830380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:37.872055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:37.942494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f7b2c-8cee-4da6-be88-5dbc4de0d066 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:37.983448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9e92e-f601-4147-8643-f5e18688ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:37.984530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9e92e-f601-4147-8643-f5e18688ff0e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:37.985178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9e92e-f601-4147-8643-f5e18688ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:37.985603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9e92e-f601-4147-8643-f5e18688ff0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:37.998857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9e92e-f601-4147-8643-f5e18688ff0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9bb9e92e-f601-4147-8643-f5e18688ff0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:38.009853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:38.010757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:38.011537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:38.014456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 63/116] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:41:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:38.015838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35969f-1fb3-4b26-8ac3-9a59caef220a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.016514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35969f-1fb3-4b26-8ac3-9a59caef220a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.017049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35969f-1fb3-4b26-8ac3-9a59caef220a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.018543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 64/117] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:38.019774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8d1e-afd4-4a9c-82d4-2b0f2de3b7e7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.020293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8d1e-afd4-4a9c-82d4-2b0f2de3b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.020729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8d1e-afd4-4a9c-82d4-2b0f2de3b7e7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.022151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 65/118] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:38.023091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8d1e-afd4-4a9c-82d4-2b0f2de3b7e7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.038138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf74ba-089f-43d6-a720-a812178f9143 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.076509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd82d98-b53c-484f-9bba-e3f7145168db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.077551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd82d98-b53c-484f-9bba-e3f7145168db None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.078012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd82d98-b53c-484f-9bba-e3f7145168db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.078357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd82d98-b53c-484f-9bba-e3f7145168db None admin] PATH_INFO: `/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.079742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd82d98-b53c-484f-9bba-e3f7145168db None admin] RBAC: Authorizing `identity:get_user(user_id=42b8c6ab6fbb49deaec22b8b6964d6a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:38.081316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd82d98-b53c-484f-9bba-e3f7145168db None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:38.083129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 54/119] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:37 2022] GET /identity/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:38.088892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec129eba-e6b7-45cd-8d92-ef93062f406c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.089360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec129eba-e6b7-45cd-8d92-ef93062f406c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.089717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec129eba-e6b7-45cd-8d92-ef93062f406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.107911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ec129eba-e6b7-45cd-8d92-ef93062f406c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:38.109210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec129eba-e6b7-45cd-8d92-ef93062f406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:38.121995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.150397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8d1e-afd4-4a9c-82d4-2b0f2de3b7e7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.172426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.203231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.204398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.205269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.205975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.206918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:38.207389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:38.209066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:38.209802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 55/120] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:38.214351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9941cb-c25f-4ba2-bc23-8e4710aa1090 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.214775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9941cb-c25f-4ba2-bc23-8e4710aa1090 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.215197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9941cb-c25f-4ba2-bc23-8e4710aa1090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.227137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 66/121] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:38.229334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.243541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f9941cb-c25f-4ba2-bc23-8e4710aa1090 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:38.244984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f9941cb-c25f-4ba2-bc23-8e4710aa1090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:38.317876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 56/122] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:38.319043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9941cb-c25f-4ba2-bc23-8e4710aa1090 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.347851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66934719-9a14-4c59-a8eb-94e8b3b8d3df None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.386154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.387149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.387538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.387915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.389321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:38.391071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:38.393049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 67/123] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:38.393959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.403063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.429968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9941cb-c25f-4ba2-bc23-8e4710aa1090 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.445500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.467402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.468546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.468992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.469457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.470795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:38.472401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:38.474614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 57/124] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:38.475670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.504843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a04c58-d98a-4e7a-903a-97e1bd90043a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.542040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.542695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.542995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.543278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.555295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:38.563655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:38.564628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:38.565426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 68/125] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:38.566102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.588953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7c88-fc73-450c-b964-20adb929e19f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.625707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.626880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.627385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.627979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.640042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:38.649722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:38.650750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:38.651535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:38.654098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 58/126] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:38.655257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.659893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.679409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3f704-8121-4793-b026-f0a6824b43fa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.686878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.715885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.716746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.717047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.717399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91/users/42b8c6ab6fbb49deaec22b8b6964d6a5/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.719490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] RBAC: Authorizing `identity:create_grant(project_id=7cc88b5db0244f738c4e498081179b91, user_id=42b8c6ab6fbb49deaec22b8b6964d6a5, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:38.720990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:38.740710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 69/127] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:38 2022] PUT /identity/v3/projects/7cc88b5db0244f738c4e498081179b91/users/42b8c6ab6fbb49deaec22b8b6964d6a5/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:38.742258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.780591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdc166-5595-490c-99ff-91958c818756 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.821054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.822172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.822568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.822972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.824450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:38.826238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:38.828296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 59/128] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:38.829330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.890254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f1ca-23f5-4376-813e-56b2beea52ed None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:38.926182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.940164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:38.941275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:38.942180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:38.942663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:38.943655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:38.944176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:38.945903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:38.957398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 70/129] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:38.958939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:38.996047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:38.999494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b5404-366b-47a7-9460-ea2c0bd2f435 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.052888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.053636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.054481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.054765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.056052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:39.056052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.057484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.067799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 60/130] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:39.068487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:39.090623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7cce4-1347-417a-b56c-deff208e66f4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.128228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.129501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.129990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.130464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91/users/d6b97d3592ef443885d4636f8757e02c/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.133091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=7cc88b5db0244f738c4e498081179b91, user_id=d6b97d3592ef443885d4636f8757e02c, role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.134957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.155269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 71/131] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:38 2022] PUT /identity/v3/projects/7cc88b5db0244f738c4e498081179b91/users/d6b97d3592ef443885d4636f8757e02c/roles/33a3e99f754a4bc69e665330de985b76 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:39.156313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:39.212530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b11f2-72d7-4f7a-8965-e19c73dbf668 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.263147 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:39.277365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b5e21-c052-48c4-8e87-4187ff1e1f6e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.298391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dad34-c4ae-49e2-aa52-dfe0807b8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.299566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dad34-c4ae-49e2-aa52-dfe0807b8844 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.299855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dad34-c4ae-49e2-aa52-dfe0807b8844 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.300134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dad34-c4ae-49e2-aa52-dfe0807b8844 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.301944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3dad34-c4ae-49e2-aa52-dfe0807b8844 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.303488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3dad34-c4ae-49e2-aa52-dfe0807b8844 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.310890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 61/132] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:39 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:39.312469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfb82b-5f77-43ae-9e2a-f23d721ffed5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.312770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfb82b-5f77-43ae-9e2a-f23d721ffed5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.313053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfb82b-5f77-43ae-9e2a-f23d721ffed5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.314563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 62/133] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:39.316859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fab0a7-dff7-4e75-b476-703e52a494b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.317788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fab0a7-dff7-4e75-b476-703e52a494b9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.318076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fab0a7-dff7-4e75-b476-703e52a494b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.318357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fab0a7-dff7-4e75-b476-703e52a494b9 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.320610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fab0a7-dff7-4e75-b476-703e52a494b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.322156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfb82b-5f77-43ae-9e2a-f23d721ffed5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:39.322303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fab0a7-dff7-4e75-b476-703e52a494b9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.324194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 72/134] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:39 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:39.325198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:39.326109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec93ee-6f76-4321-8520-6b263d749a1a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.326403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec93ee-6f76-4321-8520-6b263d749a1a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.326684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec93ee-6f76-4321-8520-6b263d749a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.346004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2dec93ee-6f76-4321-8520-6b263d749a1a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:39.347241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2dec93ee-6f76-4321-8520-6b263d749a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:39.424460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 73/135] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:39.425169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec93ee-6f76-4321-8520-6b263d749a1a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:39.535297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfb82b-5f77-43ae-9e2a-f23d721ffed5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.557661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:39.581491 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec93ee-6f76-4321-8520-6b263d749a1a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.603676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.604552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.604834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.605106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.606325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.607782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.609696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 63/136] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:39 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:39.610306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:39.624414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.625338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.625735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.626023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] PATH_INFO: `/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.628168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] RBAC: Authorizing `identity:get_project(project_id=9c35dc6046ec4f808d7e567c8eaaac5a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.629684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.631969 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 74/137] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:39 2022] GET /identity/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:39.632746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:39.726342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:39.769911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c478b-fcf2-4693-a0a6-e26b43034beb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.807637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304050c4-2c48-4bd0-8d31-6e0115d93d2e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:39.815369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.816591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.817056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.817535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91/users/d6b97d3592ef443885d4636f8757e02c/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.819909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] RBAC: Authorizing `identity:create_grant(project_id=7cc88b5db0244f738c4e498081179b91, user_id=d6b97d3592ef443885d4636f8757e02c, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.821675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.859088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 75/138] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:39 2022] PUT /identity/v3/projects/7cc88b5db0244f738c4e498081179b91/users/d6b97d3592ef443885d4636f8757e02c/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:39.859918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:39.864354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:39.865200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:39.865495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:39.865899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:39.874237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:39.876130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:39.878213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 64/139] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:39.885006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:39.886656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:40.074119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc196840-6cfd-4486-8785-6f5e533dc6ab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:40.093513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf238ed1-aba2-4fbf-85de-c7eb8db3506f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:40.117204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813985-4522-4596-bfb2-651f11409e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:40.118160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813985-4522-4596-bfb2-651f11409e81 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:40.118445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813985-4522-4596-bfb2-651f11409e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:40.118723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813985-4522-4596-bfb2-651f11409e81 None admin] PATH_INFO: `/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a/users/d6b97d3592ef443885d4636f8757e02c/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:40.120714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48813985-4522-4596-bfb2-651f11409e81 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c35dc6046ec4f808d7e567c8eaaac5a, user_id=d6b97d3592ef443885d4636f8757e02c, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:40.122212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48813985-4522-4596-bfb2-651f11409e81 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:40.134579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:40.135263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:40.135554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:40.135833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:40.136686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:40.138672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:40.144256 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 76/140] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:39 2022] PUT /identity/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a/users/d6b97d3592ef443885d4636f8757e02c/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:40.145895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0cdfb-3105-47e3-948f-683a6cbc07b3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:40.146397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0cdfb-3105-47e3-948f-683a6cbc07b3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:40.146932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0cdfb-3105-47e3-948f-683a6cbc07b3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:40.149511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 77/141] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:40.152234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:40.154885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6ad00-cb4a-4dd0-b6b0-95821ffc14df None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:40.155547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6ad00-cb4a-4dd0-b6b0-95821ffc14df None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:40.156042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6ad00-cb4a-4dd0-b6b0-95821ffc14df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:40.185632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-54a6ad00-cb4a-4dd0-b6b0-95821ffc14df None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:40.187272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-54a6ad00-cb4a-4dd0-b6b0-95821ffc14df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:40.196877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 65/142] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:41:39 2022] POST /identity/v3/users => generated 283 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:40.271945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 78/143] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:40.305109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:40.454335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478955a-1505-40e0-9f4f-d0422fb1048f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:40.497374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:40.498223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:40.498507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:40.498784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:40.511124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:40.519282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:40.520216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:40.521068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 66/144] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:40.526617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6ad00-cb4a-4dd0-b6b0-95821ffc14df None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:40.530048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:40.646842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6ad00-cb4a-4dd0-b6b0-95821ffc14df None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:40.684295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:40.684967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:40.685275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:40.685603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:40.686349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:40.686698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:40.688122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:40.697205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 79/145] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:40.702661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:40.762685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:40.812622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f4686-acf8-4df4-b196-730e9ca2fc25 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:40.848618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:40.849538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:40.849823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:40.850102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] PATH_INFO: `/v3/users/52e2cacded774b22965a2814a91a511e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:40.851978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] RBAC: Authorizing `identity:get_user(user_id=52e2cacded774b22965a2814a91a511e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:40.853468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:40.855344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 67/146] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:40 2022] GET /identity/v3/users/52e2cacded774b22965a2814a91a511e => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:40.860566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:40.965652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:41.016389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8009-ed61-4167-890f-e154c9756056 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:41.064260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8897530-4988-473b-be23-8ee16a534408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:41.065139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8897530-4988-473b-be23-8ee16a534408 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:41.065450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8897530-4988-473b-be23-8ee16a534408 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:41.065727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8897530-4988-473b-be23-8ee16a534408 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:41.066962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8897530-4988-473b-be23-8ee16a534408 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:41.068426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8897530-4988-473b-be23-8ee16a534408 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:41.070692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 80/147] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:40 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:41.076243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:41.202840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e16bed-c163-4130-aabf-7b0c76dca9b6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:41.230850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:41.239146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefabbc3-d7ef-4039-8b36-4d90a52b5183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:41.240331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabbc3-d7ef-4039-8b36-4d90a52b5183 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:41.240617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabbc3-d7ef-4039-8b36-4d90a52b5183 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:41.240891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabbc3-d7ef-4039-8b36-4d90a52b5183 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91/users/52e2cacded774b22965a2814a91a511e/roles/3208335c205449e6998150af5b78eaa2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:41.242991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefabbc3-d7ef-4039-8b36-4d90a52b5183 None admin] RBAC: Authorizing `identity:create_grant(project_id=7cc88b5db0244f738c4e498081179b91, user_id=52e2cacded774b22965a2814a91a511e, role_id=3208335c205449e6998150af5b78eaa2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:41.244442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefabbc3-d7ef-4039-8b36-4d90a52b5183 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:41.261552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 68/148] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:41 2022] PUT /identity/v3/projects/7cc88b5db0244f738c4e498081179b91/users/52e2cacded774b22965a2814a91a511e/roles/3208335c205449e6998150af5b78eaa2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:42.067010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085d737-1a36-45b0-b6c3-3c8f1481594c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:42.067513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085d737-1a36-45b0-b6c3-3c8f1481594c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:42.067585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085d737-1a36-45b0-b6c3-3c8f1481594c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:42.069269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 81/149] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:42.074486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71702e8-446c-471e-8f0f-aa48d11eb77b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:42.075282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71702e8-446c-471e-8f0f-aa48d11eb77b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:42.075713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71702e8-446c-471e-8f0f-aa48d11eb77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:42.099660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f71702e8-446c-471e-8f0f-aa48d11eb77b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:42.101790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f71702e8-446c-471e-8f0f-aa48d11eb77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:42.187466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 69/150] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:42.226733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085d737-1a36-45b0-b6c3-3c8f1481594c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:42.363648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085d737-1a36-45b0-b6c3-3c8f1481594c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:42.402304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:42.403109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:42.403393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:42.403672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:42.406274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:42.422108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:42.430303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:42.431236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:42.432055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 82/151] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:42.438527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71702e8-446c-471e-8f0f-aa48d11eb77b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:42.570106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71702e8-446c-471e-8f0f-aa48d11eb77b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:42.585862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:42.614272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46be722-1883-4732-b790-98299b9d07c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:42.615125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be722-1883-4732-b790-98299b9d07c5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:42.615821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be722-1883-4732-b790-98299b9d07c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:42.616172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be722-1883-4732-b790-98299b9d07c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:42.616971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46be722-1883-4732-b790-98299b9d07c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:42.617384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46be722-1883-4732-b790-98299b9d07c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:42.618779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46be722-1883-4732-b790-98299b9d07c5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:42.629583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 70/152] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:42.635663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:42.780604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2156-39d9-4f75-b500-9a28bc2a5773 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:42.780902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2156-39d9-4f75-b500-9a28bc2a5773 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:42.781217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2156-39d9-4f75-b500-9a28bc2a5773 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:42.782627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 71/153] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:42.786998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e90cc1-1290-4e17-ad66-4f943fd41350 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:42.795677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f91a6-b621-4823-b173-157e8220ed08 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:42.795976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f91a6-b621-4823-b173-157e8220ed08 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:42.796263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f91a6-b621-4823-b173-157e8220ed08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:42.827137 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-322f91a6-b621-4823-b173-157e8220ed08 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:42.828805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-322f91a6-b621-4823-b173-157e8220ed08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:42.850723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:42.851605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:42.851881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:42.852148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] PATH_INFO: `/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:42.854941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] RBAC: Authorizing `identity:get_user(user_id=42b8c6ab6fbb49deaec22b8b6964d6a5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:42.856435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:42.858312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 83/154] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:42 2022] GET /identity/v3/users/42b8c6ab6fbb49deaec22b8b6964d6a5 => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:42.867186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:42.943796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 72/155] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:42.984289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:43.008530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f91a6-b621-4823-b173-157e8220ed08 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:43.049831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4379d4c-0587-4e49-b7a4-32b21a35a5ce None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:43.127543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:43.128136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:43.128390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:43.128659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:43.180067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f91a6-b621-4823-b173-157e8220ed08 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:43.183201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:43.252640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:43.253493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:43.253830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:43.254114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:43.270645 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:43.283217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:43.284332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:43.285235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 73/156] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:43.292495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:43.322303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:43.323828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:43.447409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f48cc-06b2-4862-94bc-261458808b3c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:43.495732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:43.497904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:43.498604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:43.498892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:43.499174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:43.499939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:43.500250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:43.507294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:43.516987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 74/157] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:41:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:43.530508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:43.557307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 84/158] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:43.566148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:43.574037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:43.693499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74219df2-444e-413f-ba55-a730e898a928 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:43.731977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66adc-d087-4119-997d-aa5558447fc6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:43.736559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:43.737751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:43.738241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:43.738713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:43.740162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:43.745173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:43.747417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 75/159] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:43 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:43.760761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:43.805934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:43.806835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:43.807125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:43.807373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:43.808669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:43.813284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:43.815499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 85/160] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:43 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:43.822673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:43.873968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:43.966365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a8959-ca0b-4154-821d-e633a554d821 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:44.021058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e1631-1b51-4590-b678-63cc5b3b885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:44.021696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e1631-1b51-4590-b678-63cc5b3b885e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.021982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e1631-1b51-4590-b678-63cc5b3b885e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.022253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e1631-1b51-4590-b678-63cc5b3b885e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.023703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e1631-1b51-4590-b678-63cc5b3b885e None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=42b8c6ab6fbb49deaec22b8b6964d6a5) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:44.023819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:44.024022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e1631-1b51-4590-b678-63cc5b3b885e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:44.025557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e1631-1b51-4590-b678-63cc5b3b885e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:44.039653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975e584-3ce0-4e56-8994-50b4202a6df8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:44.040266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 86/161] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:43 2022] GET /identity/v3/role_assignments?user.id=42b8c6ab6fbb49deaec22b8b6964d6a5&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:44.041822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2825721-c782-414f-8547-15b8a943f832 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.042117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2825721-c782-414f-8547-15b8a943f832 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.042398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2825721-c782-414f-8547-15b8a943f832 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.044149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 87/162] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:44.049862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983ff03-5262-4bfa-a954-bec6249b13d0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.050130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983ff03-5262-4bfa-a954-bec6249b13d0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.050411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983ff03-5262-4bfa-a954-bec6249b13d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.084525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7983ff03-5262-4bfa-a954-bec6249b13d0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:44.086336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7983ff03-5262-4bfa-a954-bec6249b13d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:44.126644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705448ae-88e2-4595-b1a7-a8b034bb45de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:44.127305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705448ae-88e2-4595-b1a7-a8b034bb45de None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.127586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705448ae-88e2-4595-b1a7-a8b034bb45de None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.127855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705448ae-88e2-4595-b1a7-a8b034bb45de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.222799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 88/163] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:44.280405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983ff03-5262-4bfa-a954-bec6249b13d0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:44.284793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705448ae-88e2-4595-b1a7-a8b034bb45de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:44.286318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705448ae-88e2-4595-b1a7-a8b034bb45de None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:44.322478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:44.436068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983ff03-5262-4bfa-a954-bec6249b13d0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:44.503490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31b91f-8ca1-4dbd-af17-467ba1be9092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:44.504317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31b91f-8ca1-4dbd-af17-467ba1be9092 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.504598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31b91f-8ca1-4dbd-af17-467ba1be9092 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.504873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31b91f-8ca1-4dbd-af17-467ba1be9092 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.526410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31b91f-8ca1-4dbd-af17-467ba1be9092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0f31b91f-8ca1-4dbd-af17-467ba1be9092 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:44.535588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:44.536495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:44.537325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:44.541811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 89/164] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:44.542922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681799-20bf-40f9-9478-ed4a2938a53f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.543372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681799-20bf-40f9-9478-ed4a2938a53f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.543908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681799-20bf-40f9-9478-ed4a2938a53f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.547164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:44.548355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 90/165] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:44.549802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf681799-20bf-40f9-9478-ed4a2938a53f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:44.561363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 76/166] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 801 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:44.562697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c26f-88e2-4eca-be3d-09a5301fcae0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.563154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c26f-88e2-4eca-be3d-09a5301fcae0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.563643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c26f-88e2-4eca-be3d-09a5301fcae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.602201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6d73c26f-88e2-4eca-be3d-09a5301fcae0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:44.603402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6d73c26f-88e2-4eca-be3d-09a5301fcae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:44.681509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 77/167] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:44.682640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c26f-88e2-4eca-be3d-09a5301fcae0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:44.716441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:44.750965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf681799-20bf-40f9-9478-ed4a2938a53f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:44.808323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:44.809449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.809921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.810415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.811336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:44.811822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:44.812930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c26f-88e2-4eca-be3d-09a5301fcae0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:44.817871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:44.829825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 91/168] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:44.831047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:44.838658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:44.850941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:44.852122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:44.852426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:44.852767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:44.854077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:44.855634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:44.857652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 78/169] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:44 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:44.858510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:44.970226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8238d-8aa5-410b-988b-987ea6f51b44 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.005793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:45.007340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.008047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.011218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:45.012711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:45.017186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 79/170] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:44 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:45.017186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:45.071676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a6f3b-480d-4e62-84f7-8418e1c12fbf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.085482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:45.128747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a704f2-3bd2-4d98-979e-0db2281de625 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.168140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:45.168818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.169103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.169433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.169709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:45.170290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] RBAC: Adding query filter params (role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=d6b97d3592ef443885d4636f8757e02c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:45.170601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:45.171091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.171542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.172079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.172186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:45.180062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 80/171] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:45 2022] GET /identity/v3/role_assignments?user.id=d6b97d3592ef443885d4636f8757e02c&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=33a3e99f754a4bc69e665330de985b76 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:45.180940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:45.195890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:45.216823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:45.217800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:45.218604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:45.221052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 92/172] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:45.227751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:45.249545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:45.306018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aef11-5f88-471a-b2c7-8f2302abe724 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.358511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:45.390203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9435c-6890-4ec7-8974-9d0058c5a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:45.391180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9435c-6890-4ec7-8974-9d0058c5a87a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.391638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9435c-6890-4ec7-8974-9d0058c5a87a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.392147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9435c-6890-4ec7-8974-9d0058c5a87a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.506683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68719ce4-f6ff-493f-afec-2d155ad31fed None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.534017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9435c-6890-4ec7-8974-9d0058c5a87a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:45.535498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9435c-6890-4ec7-8974-9d0058c5a87a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:45.551719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb24291-3405-4113-ac2f-3ad3c3ec24ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:45.554450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24291-3405-4113-ac2f-3ad3c3ec24ec None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.554909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24291-3405-4113-ac2f-3ad3c3ec24ec None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.556197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24291-3405-4113-ac2f-3ad3c3ec24ec None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.558627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb24291-3405-4113-ac2f-3ad3c3ec24ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:45.559098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb24291-3405-4113-ac2f-3ad3c3ec24ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:45.561520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb24291-3405-4113-ac2f-3ad3c3ec24ec None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:45.577722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 93/173] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:45.580180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce313-5d07-45c8-afe4-ec8089f9170e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.580646 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce313-5d07-45c8-afe4-ec8089f9170e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.581790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce313-5d07-45c8-afe4-ec8089f9170e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.583451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 94/174] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:45.585696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce313-5d07-45c8-afe4-ec8089f9170e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:45.699808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:45.707916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 81/175] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:45.709511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f1d9f-ff53-48e9-8ecd-f2d21b99c279 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.709987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f1d9f-ff53-48e9-8ecd-f2d21b99c279 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.710436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f1d9f-ff53-48e9-8ecd-f2d21b99c279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.731618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-588f1d9f-ff53-48e9-8ecd-f2d21b99c279 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:45.733307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-588f1d9f-ff53-48e9-8ecd-f2d21b99c279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:45.744838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce313-5d07-45c8-afe4-ec8089f9170e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.758440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:45.786132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e06447-a989-4bbc-88be-4d9f6e045360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:45.787309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06447-a989-4bbc-88be-4d9f6e045360 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.787737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06447-a989-4bbc-88be-4d9f6e045360 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.788670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e06447-a989-4bbc-88be-4d9f6e045360 None admin] PATH_INFO: `/v3/users/d6b97d3592ef443885d4636f8757e02c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.790203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e06447-a989-4bbc-88be-4d9f6e045360 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b97d3592ef443885d4636f8757e02c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:45.794202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e06447-a989-4bbc-88be-4d9f6e045360 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:45.796576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 95/176] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:45 2022] GET /identity/v3/users/d6b97d3592ef443885d4636f8757e02c => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:45.798523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec0924-9305-4ad6-a732-360d84478028 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.799291 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec0924-9305-4ad6-a732-360d84478028 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.799705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec0924-9305-4ad6-a732-360d84478028 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.801335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 96/177] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:45.802363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec0924-9305-4ad6-a732-360d84478028 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:45.834636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 82/178] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:45.836076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f1d9f-ff53-48e9-8ecd-f2d21b99c279 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:45.941121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec0924-9305-4ad6-a732-360d84478028 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.981372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f1d9f-ff53-48e9-8ecd-f2d21b99c279 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:45.992062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c1354-e0ca-46c1-a181-40820d6cb621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:45.993444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c1354-e0ca-46c1-a181-40820d6cb621 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:45.993736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c1354-e0ca-46c1-a181-40820d6cb621 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:45.994019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c1354-e0ca-46c1-a181-40820d6cb621 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:45.995964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c1354-e0ca-46c1-a181-40820d6cb621 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:45.997749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c1354-e0ca-46c1-a181-40820d6cb621 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.000365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 97/179] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:45 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:46.002323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6ad39-3654-42b5-a43c-2229df7f3329 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.002618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6ad39-3654-42b5-a43c-2229df7f3329 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.002901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6ad39-3654-42b5-a43c-2229df7f3329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.012059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.023184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.024102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.024521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.024931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.026721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4fd6ad39-3654-42b5-a43c-2229df7f3329 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:46.027907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4fd6ad39-3654-42b5-a43c-2229df7f3329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:46.057843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.108588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 98/180] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:46.109509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6ad39-3654-42b5-a43c-2229df7f3329 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.118666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:46.120791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.237950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 83/181] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:46.238725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.239791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6ad39-3654-42b5-a43c-2229df7f3329 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.277919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea42184-1614-4751-9596-fd2f677b32bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.279053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea42184-1614-4751-9596-fd2f677b32bf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.279527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea42184-1614-4751-9596-fd2f677b32bf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.279990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea42184-1614-4751-9596-fd2f677b32bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.282237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.292966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea42184-1614-4751-9596-fd2f677b32bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:46.296987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7ea42184-1614-4751-9596-fd2f677b32bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:46.302838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:46.303804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:46.304667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 99/182] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:46.305664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796dca3-75bf-4c72-9417-174f8df37f62 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.305664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796dca3-75bf-4c72-9417-174f8df37f62 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.305957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796dca3-75bf-4c72-9417-174f8df37f62 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.307094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 100/183] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:46.307775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796dca3-75bf-4c72-9417-174f8df37f62 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.346798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da5336-a0b6-46b5-a063-3b1c13d82490 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.379424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.380080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.380357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.380629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.381490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=d6b97d3592ef443885d4636f8757e02c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:46.382314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:46.383736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.389527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 84/184] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/role_assignments?user.id=d6b97d3592ef443885d4636f8757e02c&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:46.390297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.412493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796dca3-75bf-4c72-9417-174f8df37f62 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.452033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec99b0-dab3-4252-99cd-021d57925d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.453249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec99b0-dab3-4252-99cd-021d57925d46 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.453726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec99b0-dab3-4252-99cd-021d57925d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.454296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec99b0-dab3-4252-99cd-021d57925d46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.467892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec99b0-dab3-4252-99cd-021d57925d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-aaec99b0-dab3-4252-99cd-021d57925d46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:46.477881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:46.478917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:46.479828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:46.482011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 101/185] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:46.483506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ba84-b359-481e-96d3-1170c70fa99e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.483979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ba84-b359-481e-96d3-1170c70fa99e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.484265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ba84-b359-481e-96d3-1170c70fa99e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.489304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 102/186] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:46.489304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342ba84-b359-481e-96d3-1170c70fa99e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.506565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8030ea-5f5e-468c-b348-20d7398c8f29 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.525881 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.546065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec9a31-446c-403b-960f-964df76901d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.547334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec9a31-446c-403b-960f-964df76901d3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.547614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec9a31-446c-403b-960f-964df76901d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.547900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec9a31-446c-403b-960f-964df76901d3 None admin] PATH_INFO: `/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.549502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec9a31-446c-403b-960f-964df76901d3 None admin] RBAC: Authorizing `identity:get_project(project_id=9c35dc6046ec4f808d7e567c8eaaac5a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:46.550966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec9a31-446c-403b-960f-964df76901d3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.552909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 85/187] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/projects/9c35dc6046ec4f808d7e567c8eaaac5a => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:46.554117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88e1a9-9292-455f-bbfa-ade1baac90e4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.554406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88e1a9-9292-455f-bbfa-ade1baac90e4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.554681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88e1a9-9292-455f-bbfa-ade1baac90e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.563560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.573638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6b88e1a9-9292-455f-bbfa-ade1baac90e4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:46.574859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b88e1a9-9292-455f-bbfa-ade1baac90e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:46.625807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342ba84-b359-481e-96d3-1170c70fa99e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.661536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 86/188] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:46.661722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869cd98-d78e-4406-9382-1d34babaaa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.662384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869cd98-d78e-4406-9382-1d34babaaa2f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.662485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88e1a9-9292-455f-bbfa-ade1baac90e4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.662741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869cd98-d78e-4406-9382-1d34babaaa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.663010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869cd98-d78e-4406-9382-1d34babaaa2f None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.663761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869cd98-d78e-4406-9382-1d34babaaa2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:46.664072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869cd98-d78e-4406-9382-1d34babaaa2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:46.666059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869cd98-d78e-4406-9382-1d34babaaa2f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.674862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 103/189] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:46.676062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ee1ad-4947-4585-a895-874a0c3e475a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.676355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ee1ad-4947-4585-a895-874a0c3e475a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.676632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ee1ad-4947-4585-a895-874a0c3e475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.693672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ae1ee1ad-4947-4585-a895-874a0c3e475a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:46.695485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae1ee1ad-4947-4585-a895-874a0c3e475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:46.760774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 104/190] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:46.761849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ee1ad-4947-4585-a895-874a0c3e475a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.778031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88e1a9-9292-455f-bbfa-ade1baac90e4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.794502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.807140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:46.813830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.814504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.814784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.815058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.815806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:46.816114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:46.817528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.826470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 87/191] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:46.827126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.863444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ee1ad-4947-4585-a895-874a0c3e475a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.898316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.898971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.899261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.899654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.900662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=9c35dc6046ec4f808d7e567c8eaaac5a, user.id=d6b97d3592ef443885d4636f8757e02c) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:46.901195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:46.903121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.909051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 105/192] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/role_assignments?user.id=d6b97d3592ef443885d4636f8757e02c&scope.project.id=9c35dc6046ec4f808d7e567c8eaaac5a&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:46.914091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:46.927941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a81715-0e5b-4d3a-965c-2bb6011c4849 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:46.968249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:46.969531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:46.969840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:46.970088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] PATH_INFO: `/v3/users/52e2cacded774b22965a2814a91a511e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:46.971449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] RBAC: Authorizing `identity:get_user(user_id=52e2cacded774b22965a2814a91a511e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:46.972971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:46.974963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 88/193] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/users/52e2cacded774b22965a2814a91a511e => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:46.980797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.032092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.035492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16d02c-e008-42ea-bdef-283a9f617958 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.048428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.077019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.077889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.078179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.078458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.091769 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:47.092072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390b3f0-f828-4db9-8c64-85b9690cfff8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:47.099794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:47.100752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:47.101609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 106/194] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:47.102976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.132811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.133751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.134033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.134310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.135497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.136948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.138623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 89/195] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:47.139399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.219283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc26b84-45d8-4189-ae0d-ff947834cbf8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.250733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff11a-7515-40a7-a5e4-8845c720f691 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.252656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.253590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.253872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.254153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] PATH_INFO: `/v3/users/90f4d985aed94393930a1be372da437c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.265129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.267971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] RBAC: Authorizing `identity:get_user(user_id=90f4d985aed94393930a1be372da437c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.269709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.271574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 107/196] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/users/90f4d985aed94393930a1be372da437c => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:47.272147 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.286989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.287575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.287890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.288163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.300352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.356387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.357895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.374564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024126d-c412-41f9-b025-f7f2be0c5943 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.407688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.408360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.408647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.408926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.409738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:47.410044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.411413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.419304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 108/197] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:47.420068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.459501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 90/198] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:47.460368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.476634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.513949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.524116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a85f6b-4210-41e9-8b17-cf2bddeb5524 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.557865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.558524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.558811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.559090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.559798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.560900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6207e-526e-4756-a960-4ee5d67dde04 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.561282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.580711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 109/199] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:41:47 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:47.581378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.600227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.600950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.601271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.601544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.677656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.679750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.697727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.708518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c012b-5481-4ddb-a37d-ca844b4af665 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.751057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.751905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.752192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.752466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] PATH_INFO: `/v3/users/90f4d985aed94393930a1be372da437c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.753743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] RBAC: Authorizing `identity:get_user(user_id=90f4d985aed94393930a1be372da437c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.755222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.757459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 110/200] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/users/90f4d985aed94393930a1be372da437c => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:47.758225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.779844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.796639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 91/201] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:47.797743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.885676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f82a01-633f-4aeb-a598-f08520e4bda9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.926451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.927388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.927677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.927973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:47.929228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:47.930695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:47.932545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 111/202] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:47.932768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e1fd2-3202-4472-9511-e38feda845d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:47.933175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:47.949892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:47.974971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:47.975845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:47.976131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:47.976402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:48.032153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:48.047518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:48.049279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:48.050939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7e7b-46d4-4bcd-9d1c-23ab6bc4bf3d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:48.091286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:48.092166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:48.092563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:48.092846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:48.094075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:48.095632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:48.097572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 112/203] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:48.098421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:48.162094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 92/204] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:48.163065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:48.191275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:48.229113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa9104-b0ac-44d8-aa80-ba6ecc217f85 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:48.265246 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:48.265920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:48.266208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:48.266484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:48.267319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] RBAC: Adding query filter params (role.id=3208335c205449e6998150af5b78eaa2, scope.project.id=7cc88b5db0244f738c4e498081179b91, user.id=52e2cacded774b22965a2814a91a511e) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:48.267629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:48.269222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:48.277089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 113/205] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:48 2022] GET /identity/v3/role_assignments?user.id=52e2cacded774b22965a2814a91a511e&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=3208335c205449e6998150af5b78eaa2 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:48.285210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:48.306403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed6bd8-8623-47ee-9e95-e862da5ef44d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:48.318262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:48.367194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:48.367901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:48.368143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:48.368433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:48.369302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=90f4d985aed94393930a1be372da437c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:48.369615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:48.371080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:48.384463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 93/206] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:48 2022] GET /identity/v3/role_assignments?user.id=90f4d985aed94393930a1be372da437c&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:48.445364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02881b61-b249-4cd8-9dd5-6980e1d3560b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:48.484441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e83e5-f623-4683-bb91-48cbf3dad5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:48.485357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e83e5-f623-4683-bb91-48cbf3dad5aa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:48.485648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e83e5-f623-4683-bb91-48cbf3dad5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:48.485924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e83e5-f623-4683-bb91-48cbf3dad5aa None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:48.487160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e83e5-f623-4683-bb91-48cbf3dad5aa None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:48.488640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e83e5-f623-4683-bb91-48cbf3dad5aa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:48.490824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 114/207] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:48 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:48.502366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:48.589613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:48.614469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb32b3-1d19-4a7a-ac9c-3d02a2f59c68 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:48.654170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d22b72-197b-473a-9ee9-6b19f88f6b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:48.654842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d22b72-197b-473a-9ee9-6b19f88f6b38 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:48.655124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d22b72-197b-473a-9ee9-6b19f88f6b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:48.655400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d22b72-197b-473a-9ee9-6b19f88f6b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:48.656250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d22b72-197b-473a-9ee9-6b19f88f6b38 None admin] RBAC: Adding query filter params (role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=90f4d985aed94393930a1be372da437c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:48.656634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d22b72-197b-473a-9ee9-6b19f88f6b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:48.658148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d22b72-197b-473a-9ee9-6b19f88f6b38 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:48.665473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 94/208] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:48 2022] GET /identity/v3/role_assignments?user.id=90f4d985aed94393930a1be372da437c&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=33a3e99f754a4bc69e665330de985b76 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:50.036390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec9669-11c7-485d-9283-673eeb1cdb5f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.036903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec9669-11c7-485d-9283-673eeb1cdb5f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.036976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec9669-11c7-485d-9283-673eeb1cdb5f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.038898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 115/209] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:50.044187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c0cbe-1b4f-420c-8ca4-3f62df1104f2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.044598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c0cbe-1b4f-420c-8ca4-3f62df1104f2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.044765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c0cbe-1b4f-420c-8ca4-3f62df1104f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.062543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8f3c0cbe-1b4f-420c-8ca4-3f62df1104f2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:50.063722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f3c0cbe-1b4f-420c-8ca4-3f62df1104f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:50.102245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81df8a-06e9-46ee-8f2f-2d9a0a97fa0d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.102550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81df8a-06e9-46ee-8f2f-2d9a0a97fa0d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.102796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81df8a-06e9-46ee-8f2f-2d9a0a97fa0d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.104309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 116/210] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:50.109086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdfed9-23ee-408d-87c5-350aebf2057e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.109399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdfed9-23ee-408d-87c5-350aebf2057e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.109685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdfed9-23ee-408d-87c5-350aebf2057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.127252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-96cdfed9-23ee-408d-87c5-350aebf2057e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:50.128424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-96cdfed9-23ee-408d-87c5-350aebf2057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:50.145543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 95/211] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:50.170210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c0cbe-1b4f-420c-8ca4-3f62df1104f2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:50.213041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 117/212] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:50.237656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdfed9-23ee-408d-87c5-350aebf2057e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:50.253591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:50.261519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:50.286573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c0cbe-1b4f-420c-8ca4-3f62df1104f2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:50.324102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:50.325273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.325770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.326236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.327618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:50.329484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:50.336537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 96/213] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:50.337647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:50.349568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdfed9-23ee-408d-87c5-350aebf2057e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:50.387922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:50.388735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.389018 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.389378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.400202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:50.409132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:50.410049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:50.410852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 118/214] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:50.417035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:50.455698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d00137-2733-44fa-bb8e-6ec20e0a6a8e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:50.493349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:50.494366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.494930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.495633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:50.496193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.497110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:50.498768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:50.511106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:50.533835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 97/215] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:41:50 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:50.534312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad167e-d2b8-4e45-8f2b-aeaeb964df91 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:50.591924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb839ce-9273-4905-b722-2fb9806bf427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:50.592604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb839ce-9273-4905-b722-2fb9806bf427 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.592891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb839ce-9273-4905-b722-2fb9806bf427 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.593260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb839ce-9273-4905-b722-2fb9806bf427 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.594048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb839ce-9273-4905-b722-2fb9806bf427 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:50.594401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb839ce-9273-4905-b722-2fb9806bf427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:50.595822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb839ce-9273-4905-b722-2fb9806bf427 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:50.607294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 119/216] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:50.612928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:50.727616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25e0e-74a4-4306-ae66-3329c58b09f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:50.741630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eb29c-7d43-4f94-b412-26ce8826fd93 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.742000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eb29c-7d43-4f94-b412-26ce8826fd93 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.742202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eb29c-7d43-4f94-b412-26ce8826fd93 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.743827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 120/217] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:50.749483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032d1c-5b21-44f2-9809-b096f10d540d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.749784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032d1c-5b21-44f2-9809-b096f10d540d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.750049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032d1c-5b21-44f2-9809-b096f10d540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.768374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:50.769273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.769554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.769829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] PATH_INFO: `/v3/users/330dd838535b48f5aac96898aa501b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:50.770917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-82032d1c-5b21-44f2-9809-b096f10d540d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:50.772332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-82032d1c-5b21-44f2-9809-b096f10d540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:50.789987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] RBAC: Authorizing `identity:get_user(user_id=330dd838535b48f5aac96898aa501b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:50.791490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:50.793284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 98/218] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:50 2022] GET /identity/v3/users/330dd838535b48f5aac96898aa501b6c => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:50.799465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:50.856871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 121/219] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:50.867808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:50.881075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032d1c-5b21-44f2-9809-b096f10d540d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:50.921931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878886c-532e-40ed-b1bd-d0ccbec4703a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:50.965194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:50.967679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1ecb6-cdba-4aa2-be44-0583ab5e2707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:50.967814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ecb6-cdba-4aa2-be44-0583ab5e2707 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:50.968102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ecb6-cdba-4aa2-be44-0583ab5e2707 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:50.968351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ecb6-cdba-4aa2-be44-0583ab5e2707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.010733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032d1c-5b21-44f2-9809-b096f10d540d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.047560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ecb27-7ac1-4cbe-8291-fcb5c3de7f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.049556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ecb27-7ac1-4cbe-8291-fcb5c3de7f55 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.050049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ecb27-7ac1-4cbe-8291-fcb5c3de7f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.050527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ecb27-7ac1-4cbe-8291-fcb5c3de7f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.062252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ecb27-7ac1-4cbe-8291-fcb5c3de7f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:51.062962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ecb6-cdba-4aa2-be44-0583ab5e2707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:51.064454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ecb6-cdba-4aa2-be44-0583ab5e2707 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-516ecb27-7ac1-4cbe-8291-fcb5c3de7f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:51.071392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:51.072330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:51.073146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:51.075876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 122/220] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:51.082667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a4768-514f-4996-ac3e-c5d3b9febb70 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.083207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a4768-514f-4996-ac3e-c5d3b9febb70 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.083743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a4768-514f-4996-ac3e-c5d3b9febb70 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.086285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 123/221] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:51.087755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a4768-514f-4996-ac3e-c5d3b9febb70 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.127307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:51.188776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 99/222] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:41:51.188776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bef23-9c16-4170-90ae-47203e648a18 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.188776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bef23-9c16-4170-90ae-47203e648a18 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.188776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bef23-9c16-4170-90ae-47203e648a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.206885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a4768-514f-4996-ac3e-c5d3b9febb70 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.211278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f75bef23-9c16-4170-90ae-47203e648a18 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:51.212674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f75bef23-9c16-4170-90ae-47203e648a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:51.233075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:51.246714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.247559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.247964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.248345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.249440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:51.249870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:51.251704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:51.261316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 124/223] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:51.263398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.289218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 100/224] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:51.291168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bef23-9c16-4170-90ae-47203e648a18 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.379202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e198f8-bb7d-44d7-8e18-f00062b530f9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.395924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:51.414864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bef23-9c16-4170-90ae-47203e648a18 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.419547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.420426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.420707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.420981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.422231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:51.424041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:51.426051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 125/225] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:51.426697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.460330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.461494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.461783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.462058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] PATH_INFO: `/v3/users/90f4d985aed94393930a1be372da437c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.464013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] RBAC: Authorizing `identity:get_user(user_id=90f4d985aed94393930a1be372da437c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:51.466297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:51.469196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 101/226] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/users/90f4d985aed94393930a1be372da437c => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:51.470076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.505804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:51.557942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdd8da-635d-41ab-997d-8ada68aed443 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.595892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.596832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.597217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.597884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.608687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:51.612382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5bd79-6062-4a7b-8c88-e32b8898d652 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:51.617560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:51.618643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:51.619519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 126/227] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:51.620193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.661400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.662303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.662772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.663187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.664163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=330dd838535b48f5aac96898aa501b6c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:51.665179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:51.667264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:51.675345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 102/228] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/role_assignments?user.id=330dd838535b48f5aac96898aa501b6c&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:51.676289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.699129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:51.801806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424c489-a2cc-4366-8a85-08b02924af2e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.820946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38111f-64b4-42e5-b92d-9f36627a31b7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:51.845331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.846275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.846706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.847155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.848084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:51.848544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:51.851059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:51.851235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:51.862352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 103/229] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:51.870245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.914146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:51.915510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:51.915510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:51.915510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:51.916896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:51.918465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:51.920339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 127/230] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:51.926535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:51.981391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:52.010934 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ce292-cfc3-4632-8570-1472079d4c83 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:52.053119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:52.054592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:52.054891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:52.055154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] PATH_INFO: `/v3/users/90f4d985aed94393930a1be372da437c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:52.060847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] RBAC: Authorizing `identity:get_user(user_id=90f4d985aed94393930a1be372da437c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:52.060847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:52.061190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 104/231] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:51 2022] GET /identity/v3/users/90f4d985aed94393930a1be372da437c => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:52.062912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db77455-ea28-4983-afb1-9872e6c2c1aa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:52.070562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:52.097934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:52.098925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:52.099208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:52.099576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/90f4d985aed94393930a1be372da437c/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:52.101640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8a47ac1834033aef7a1d6151bd94e, user_id=90f4d985aed94393930a1be372da437c, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:52.103152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:52.119276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 128/232] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:51 2022] PUT /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/90f4d985aed94393930a1be372da437c/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:41:52.202004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a053-fe8f-4dc6-aecb-f025e64069ec None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:52.243080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbb9ae-9017-428f-beec-7d1f441e0273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:52.243969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbb9ae-9017-428f-beec-7d1f441e0273 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:52.244257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbb9ae-9017-428f-beec-7d1f441e0273 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:52.244536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbb9ae-9017-428f-beec-7d1f441e0273 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:52.245777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbb9ae-9017-428f-beec-7d1f441e0273 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:52.247254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbb9ae-9017-428f-beec-7d1f441e0273 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:52.249320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 105/233] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:52 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:52.254991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:52.330578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:52.408177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8df4a7-f775-4d8b-9084-df9d66e06a6f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:52.447862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27592a-e562-4f6f-a1e7-9861ef60c2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:52.448739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27592a-e562-4f6f-a1e7-9861ef60c2e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:52.449072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27592a-e562-4f6f-a1e7-9861ef60c2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:52.449379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27592a-e562-4f6f-a1e7-9861ef60c2e3 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/90f4d985aed94393930a1be372da437c/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:52.451602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27592a-e562-4f6f-a1e7-9861ef60c2e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8a47ac1834033aef7a1d6151bd94e, user_id=90f4d985aed94393930a1be372da437c, role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:52.453055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27592a-e562-4f6f-a1e7-9861ef60c2e3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:52.470317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 129/234] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:52 2022] PUT /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/90f4d985aed94393930a1be372da437c/roles/33a3e99f754a4bc69e665330de985b76 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:53.192893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384aeae-e316-42a4-a75b-e1e965fe1ace None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:53.193432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384aeae-e316-42a4-a75b-e1e965fe1ace None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:53.193512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384aeae-e316-42a4-a75b-e1e965fe1ace None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:53.195800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 106/235] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:53.201444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b082e-9f75-4db4-b2e3-b94ec34281a8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:53.201855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b082e-9f75-4db4-b2e3-b94ec34281a8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:53.202023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b082e-9f75-4db4-b2e3-b94ec34281a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:53.219936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7e0b082e-9f75-4db4-b2e3-b94ec34281a8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:53.221129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7e0b082e-9f75-4db4-b2e3-b94ec34281a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:53.299173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 130/236] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:53.339330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384aeae-e316-42a4-a75b-e1e965fe1ace None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:53.360217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:53.384253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f68ae-9faf-4ba0-96fa-0dba25c3a37d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:53.384589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f68ae-9faf-4ba0-96fa-0dba25c3a37d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:53.384822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f68ae-9faf-4ba0-96fa-0dba25c3a37d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:53.386531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 131/237] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:53.392098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d147ed4-212e-4a3e-93e3-ec8d10f0a304 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:53.392395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d147ed4-212e-4a3e-93e3-ec8d10f0a304 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:53.392680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d147ed4-212e-4a3e-93e3-ec8d10f0a304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:53.409819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1d147ed4-212e-4a3e-93e3-ec8d10f0a304 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:53.411015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1d147ed4-212e-4a3e-93e3-ec8d10f0a304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:53.464672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:53.478637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384aeae-e316-42a4-a75b-e1e965fe1ace None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:53.537814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:53.539060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:53.539602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:53.540070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:53.545443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 132/238] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:53.570939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:53.578814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:53.579678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:53.580548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:53.585997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 107/239] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:53.591699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d147ed4-212e-4a3e-93e3-ec8d10f0a304 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:53.594998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:53.728390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:53.778220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5733e3-62e9-4910-88c6-c445d6b5b365 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:53.808373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d147ed4-212e-4a3e-93e3-ec8d10f0a304 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:53.849381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:53.850718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:53.851005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:53.851280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:53.852781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:53.854277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:53.856055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 108/240] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:53.865166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:53.893870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:53.894564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:53.894862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:53.895131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:53.895898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:53.896211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:53.897703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:53.919329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 133/241] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:53.938513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:53.979433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a64149-d0cd-47e7-8a88-f717ae25c1c0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:54.008436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:54.041459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:54.042138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.042424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.042702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.043430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:54.044886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:54.096765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:54.112381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 109/242] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:41:53 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:54.124510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c247d-6278-4e43-9d99-e4f06565f52c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:54.164296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:54.165861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.166158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.166440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] PATH_INFO: `/v3/users/ddf3a5e2e3604552b62f84e0cfc5dda7` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.187590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf3a5e2e3604552b62f84e0cfc5dda7)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:54.189207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:54.196977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 134/243] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:53 2022] GET /identity/v3/users/ddf3a5e2e3604552b62f84e0cfc5dda7 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:54.198277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:54.316301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ff61-373d-4931-9724-a4f94452dae7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:54.358743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922ab0e-6377-4fd4-870d-817413018389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:54.359428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ab0e-6377-4fd4-870d-817413018389 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.359712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ab0e-6377-4fd4-870d-817413018389 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.359985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ab0e-6377-4fd4-870d-817413018389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.440725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:54.445468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922ab0e-6377-4fd4-870d-817413018389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:54.446947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922ab0e-6377-4fd4-870d-817413018389 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:54.552500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 110/244] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:54.558434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:54.628256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145e34-3493-4d02-a733-84993c8de32b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.628466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145e34-3493-4d02-a733-84993c8de32b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.628736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145e34-3493-4d02-a733-84993c8de32b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.630988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 111/245] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:54.637774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433602de-56ea-406d-a9de-686e9fc2018f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.638341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433602de-56ea-406d-a9de-686e9fc2018f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.638816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433602de-56ea-406d-a9de-686e9fc2018f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.656719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-433602de-56ea-406d-a9de-686e9fc2018f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:54.657666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:54.658835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-433602de-56ea-406d-a9de-686e9fc2018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:54.679009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff2226-21f7-4bc7-8825-fb4b285244b7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:54.716957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:54.718148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.718582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.719043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.720461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:54.722224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:54.724470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 135/246] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:54 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:54.730160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:54.736278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 112/247] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:54.763108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433602de-56ea-406d-a9de-686e9fc2018f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:54.792850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:54.852162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a12f5-e786-414a-a715-2ae7e3351ae5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:54.897665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433602de-56ea-406d-a9de-686e9fc2018f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:54.900561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acccc9-b279-4299-9bf0-cb0370acff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:54.901308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acccc9-b279-4299-9bf0-cb0370acff47 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.901619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acccc9-b279-4299-9bf0-cb0370acff47 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.901836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acccc9-b279-4299-9bf0-cb0370acff47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.902689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acccc9-b279-4299-9bf0-cb0370acff47 None admin] RBAC: Adding query filter params (role.id=3208335c205449e6998150af5b78eaa2, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=ddf3a5e2e3604552b62f84e0cfc5dda7) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:54.903007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acccc9-b279-4299-9bf0-cb0370acff47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:54.904510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acccc9-b279-4299-9bf0-cb0370acff47 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:54.911840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 136/248] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:54 2022] GET /identity/v3/role_assignments?user.id=ddf3a5e2e3604552b62f84e0cfc5dda7&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=3208335c205449e6998150af5b78eaa2 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:54.913302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cf166-2ba3-465e-af3a-74ef7ed66a55 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.913595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cf166-2ba3-465e-af3a-74ef7ed66a55 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.913923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cf166-2ba3-465e-af3a-74ef7ed66a55 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.915381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 137/249] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:54.924192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7e19a-f173-4d84-907c-2e123cd263ac None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.924488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7e19a-f173-4d84-907c-2e123cd263ac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.924774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7e19a-f173-4d84-907c-2e123cd263ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.940336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:54.941198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:54.941481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:54.941751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:54.946229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-48e7e19a-f173-4d84-907c-2e123cd263ac None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:54.947452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-48e7e19a-f173-4d84-907c-2e123cd263ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:54.956271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:54.960437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:54.964548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:54.965854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:54.966833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 113/250] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:41:54.973329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:55.052227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 138/251] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:55.102087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7e19a-f173-4d84-907c-2e123cd263ac None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:55.216605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:55.243989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bd49d-fa39-495b-99e0-6d2ea100539e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:55.345098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:55.345802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:55.346248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:55.346694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:55.347582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:55.348080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:55.357943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:55.368997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7e19a-f173-4d84-907c-2e123cd263ac None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:55.371660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 114/252] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:55.377316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:55.465961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:55.466706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:55.466998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:55.467282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:55.498097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:55.514673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:55.515857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:55.516650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 139/253] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:55.525281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:55.527199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581cd34-746d-48e5-adfd-350f6bb746a8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:55.532619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:55.593782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad277c2-2600-43ba-9a22-a6a3af8a5519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:55.594640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad277c2-2600-43ba-9a22-a6a3af8a5519 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:55.594920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad277c2-2600-43ba-9a22-a6a3af8a5519 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:55.595193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad277c2-2600-43ba-9a22-a6a3af8a5519 None admin] PATH_INFO: `/v3/users/330dd838535b48f5aac96898aa501b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:55.596492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad277c2-2600-43ba-9a22-a6a3af8a5519 None admin] RBAC: Authorizing `identity:get_user(user_id=330dd838535b48f5aac96898aa501b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:55.597989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad277c2-2600-43ba-9a22-a6a3af8a5519 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:55.599699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 115/254] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:55 2022] GET /identity/v3/users/330dd838535b48f5aac96898aa501b6c => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:55.606306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c61f-b5b0-45b7-bc4f-18240c5de738 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:55.606594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c61f-b5b0-45b7-bc4f-18240c5de738 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:55.606868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c61f-b5b0-45b7-bc4f-18240c5de738 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:55.615488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 116/255] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:55.616865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c61f-b5b0-45b7-bc4f-18240c5de738 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:55.658845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:55.783273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68f17d-d8cd-41b0-be2c-bef13664bfc2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:55.814631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c61f-b5b0-45b7-bc4f-18240c5de738 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:55.873184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:55.873184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:55.873184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:55.873184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:55.873585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:55.873585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:55.877173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:55.878405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4030d14-71ef-4489-8905-8f070de0bcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:55.879295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4030d14-71ef-4489-8905-8f070de0bcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:55.883380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4030d14-71ef-4489-8905-8f070de0bcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:55.883380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4030d14-71ef-4489-8905-8f070de0bcf3 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:55.883380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4030d14-71ef-4489-8905-8f070de0bcf3 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:55.883632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4030d14-71ef-4489-8905-8f070de0bcf3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:55.885601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 117/256] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:55 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:55.892156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab414d-2990-4837-a18d-6a1e3a52b95c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:55.892446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab414d-2990-4837-a18d-6a1e3a52b95c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:55.892724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab414d-2990-4837-a18d-6a1e3a52b95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:55.901386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 140/257] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:55.902277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:55.936552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-35ab414d-2990-4837-a18d-6a1e3a52b95c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:55.937766 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-35ab414d-2990-4837-a18d-6a1e3a52b95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:56.048450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:56.072806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 118/258] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:56.085175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab414d-2990-4837-a18d-6a1e3a52b95c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:56.112680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:56.199127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3abbb-b326-45cb-b5d2-1e2bbcf6cde8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:56.222523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab414d-2990-4837-a18d-6a1e3a52b95c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:56.263630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:56.264526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:56.264807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:56.265080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] PATH_INFO: `/v3/users/90f4d985aed94393930a1be372da437c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:56.266393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] RBAC: Authorizing `identity:get_user(user_id=90f4d985aed94393930a1be372da437c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:56.267901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:56.270399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 119/259] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:56 2022] GET /identity/v3/users/90f4d985aed94393930a1be372da437c => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:56.274087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:56.349977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:56.351384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:56.351672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:56.351949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/330dd838535b48f5aac96898aa501b6c/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:56.362007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8a47ac1834033aef7a1d6151bd94e, user_id=330dd838535b48f5aac96898aa501b6c, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:56.363515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:56.369569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:56.390469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b83869-c2fc-4a03-8b66-5f9ff3c0cbd9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:56.415720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 141/260] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:55 2022] PUT /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/330dd838535b48f5aac96898aa501b6c/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:56.416582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:56.450568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:56.451470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:56.451756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:56.453231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:56.479666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:56.497030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:56.498036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:56.498872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 120/261] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:41:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:56.514584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:56.650948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb97e-aed8-4c67-bbfe-9ae9966b798b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:56.735154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:56.757217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94d6ce-0653-47d6-8b4b-4a2a1bfc4b29 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:56.759339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:56.760039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:56.760325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:56.760604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:56.850313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:56.851437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:56.852911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:56.853019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:56.857856 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:56.858380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:56.859521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:56.860236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:56.864091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:56.889686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 121/262] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:41:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:56.899315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:56.925870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:56.974597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 142/263] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:56.981081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:57.012293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:57.163993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ffbca-fae3-435a-84ac-5c042a4577eb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:57.187287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b10de-bfa2-416b-a9c8-9e3ec24225c4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:57.221993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:57.222907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.224025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.224025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.224917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:57.227247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:57.229230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 143/264] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:56 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:57.242286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:57.282087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:57.282989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.283270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.283544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] PATH_INFO: `/v3/users/90f4d985aed94393930a1be372da437c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.284851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] RBAC: Authorizing `identity:get_user(user_id=90f4d985aed94393930a1be372da437c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:57.286341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:57.288536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 122/265] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:56 2022] GET /identity/v3/users/90f4d985aed94393930a1be372da437c => generated 283 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:57.302131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:57.422150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:57.438062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:57.451997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9807281-9b70-4f1a-b469-119db15e1086 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:57.461632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc7c8-19e1-43e6-b1c5-58520110545a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:57.496939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:57.497619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.497901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.498173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.516844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc166da-d210-4877-965e-7631b0e4b917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:57.517884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc166da-d210-4877-965e-7631b0e4b917 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.518376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc166da-d210-4877-965e-7631b0e4b917 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.518998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc166da-d210-4877-965e-7631b0e4b917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.520149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc166da-d210-4877-965e-7631b0e4b917 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=90f4d985aed94393930a1be372da437c) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:57.520686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc166da-d210-4877-965e-7631b0e4b917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:57.523164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc166da-d210-4877-965e-7631b0e4b917 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:57.539841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 144/266] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:57 2022] GET /identity/v3/role_assignments?user.id=90f4d985aed94393930a1be372da437c&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:57.541659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c27f-798d-4906-b993-458f1b399bfd None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.542167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c27f-798d-4906-b993-458f1b399bfd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.542688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c27f-798d-4906-b993-458f1b399bfd None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.545601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 145/267] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:57.554885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bac76-9979-43a2-a717-00d76665900c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.555572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bac76-9979-43a2-a717-00d76665900c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.556050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bac76-9979-43a2-a717-00d76665900c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.579363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:57.581000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:57.583505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dc4bac76-9979-43a2-a717-00d76665900c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:57.585604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dc4bac76-9979-43a2-a717-00d76665900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:57.672799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:57.706265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 123/268] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:41:57.714705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:57.763616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 146/269] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:57.822238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bac76-9979-43a2-a717-00d76665900c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:57.822428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0eccc-ee76-41d1-8ace-1136fb2f16aa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:57.862388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a9213-a058-4bf3-827b-b5a0629b53c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:57.863273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a9213-a058-4bf3-827b-b5a0629b53c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.863548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a9213-a058-4bf3-827b-b5a0629b53c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.863815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a9213-a058-4bf3-827b-b5a0629b53c3 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.865044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a9213-a058-4bf3-827b-b5a0629b53c3 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:57.866597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a9213-a058-4bf3-827b-b5a0629b53c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:57.868936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 124/270] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:57 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:57.874226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1679b-3139-4fb5-91a2-569f9e0e8b3a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:57.874668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1679b-3139-4fb5-91a2-569f9e0e8b3a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:57.874955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1679b-3139-4fb5-91a2-569f9e0e8b3a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:57.876589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 125/271] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:57.877280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1679b-3139-4fb5-91a2-569f9e0e8b3a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:57.937541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:57.993206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:58.003490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1679b-3139-4fb5-91a2-569f9e0e8b3a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.013365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bac76-9979-43a2-a717-00d76665900c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.047914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e75b7d-ddcf-41de-b58b-ffbd1e0cc478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.049104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e75b7d-ddcf-41de-b58b-ffbd1e0cc478 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.049442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e75b7d-ddcf-41de-b58b-ffbd1e0cc478 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.049715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e75b7d-ddcf-41de-b58b-ffbd1e0cc478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.050558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e75b7d-ddcf-41de-b58b-ffbd1e0cc478 None admin] RBAC: Adding query filter params (role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=90f4d985aed94393930a1be372da437c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:58.050865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e75b7d-ddcf-41de-b58b-ffbd1e0cc478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:58.053363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e75b7d-ddcf-41de-b58b-ffbd1e0cc478 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:58.060759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 126/272] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:41:57 2022] GET /identity/v3/role_assignments?user.id=90f4d985aed94393930a1be372da437c&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=33a3e99f754a4bc69e665330de985b76 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:58.062058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc2b9-df24-4de3-9589-7087324b4934 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.062349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc2b9-df24-4de3-9589-7087324b4934 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.062621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc2b9-df24-4de3-9589-7087324b4934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.065643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.066879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.067456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.067933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.084891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c2cfc2b9-df24-4de3-9589-7087324b4934 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:58.085729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:58.086750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c2cfc2b9-df24-4de3-9589-7087324b4934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:58.094810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:58.096370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:58.097213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 147/273] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:58.102609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:58.189192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 127/274] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:58.218268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc2b9-df24-4de3-9589-7087324b4934 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:58.242275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1b212-ac13-476b-8f3f-df0b06b0040e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.283032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4589f-5760-4caa-af99-8f8ce8e3bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.283649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4589f-5760-4caa-af99-8f8ce8e3bd32 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.283942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4589f-5760-4caa-af99-8f8ce8e3bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.284221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4589f-5760-4caa-af99-8f8ce8e3bd32 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.285130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4589f-5760-4caa-af99-8f8ce8e3bd32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:58.285482 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4589f-5760-4caa-af99-8f8ce8e3bd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:58.286093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:58.287712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4589f-5760-4caa-af99-8f8ce8e3bd32 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:58.303936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 148/275] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:58.305690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfcf94-8a0f-4c8b-8af4-426b29b45855 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.305986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfcf94-8a0f-4c8b-8af4-426b29b45855 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.306269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfcf94-8a0f-4c8b-8af4-426b29b45855 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.308093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 149/276] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:41:58.308894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfcf94-8a0f-4c8b-8af4-426b29b45855 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:58.328469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:58.363250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc2b9-df24-4de3-9589-7087324b4934 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.401039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc1c0f-e8d5-4ec7-aaf3-4f2d4f65a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.401887 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc1c0f-e8d5-4ec7-aaf3-4f2d4f65a0eb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.402167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc1c0f-e8d5-4ec7-aaf3-4f2d4f65a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.402438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc1c0f-e8d5-4ec7-aaf3-4f2d4f65a0eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.403623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc1c0f-e8d5-4ec7-aaf3-4f2d4f65a0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:58.405082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc1c0f-e8d5-4ec7-aaf3-4f2d4f65a0eb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:58.406900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 128/277] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:41:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:58.408191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c884-ac12-4334-b64b-220d9f88d809 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.408476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c884-ac12-4334-b64b-220d9f88d809 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.408747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c884-ac12-4334-b64b-220d9f88d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.427624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-af03c884-ac12-4334-b64b-220d9f88d809 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:58.428849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-af03c884-ac12-4334-b64b-220d9f88d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:58.453130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfcf94-8a0f-4c8b-8af4-426b29b45855 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.492364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.493281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.493571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.493850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] PATH_INFO: `/v3/users/5e25a5b51d424d65b8467cf141191aff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.504301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 129/278] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:41:58.505295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03c884-ac12-4334-b64b-220d9f88d809 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:58.514009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] RBAC: Authorizing `identity:get_user(user_id=5e25a5b51d424d65b8467cf141191aff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:58.516245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:58.519143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 150/279] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:58 2022] GET /identity/v3/users/5e25a5b51d424d65b8467cf141191aff => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:58.525195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:58.542748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:58.598232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:58.618109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03c884-ac12-4334-b64b-220d9f88d809 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.635740 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0b7c5-7b44-45ca-a9be-437601c0885e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.663449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.664128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.664410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.664686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.665432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:58.666879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:58.681688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.682340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.682643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.682915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.683739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, user.id=5e25a5b51d424d65b8467cf141191aff, scope.system=all) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:58.684050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:58.685523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:58.690113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 130/280] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:41:58 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:41:58.690745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:58.698839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 151/281] 10.222.0.91 () {56 vars in 1308 bytes} [Thu May 26 13:41:58 2022] GET /identity/v3/role_assignments?user.id=5e25a5b51d424d65b8467cf141191aff&scope.system=all&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:41:58.826367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:58.838231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87f95e-af81-47dc-86b7-336a9bcc5561 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:58.882006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:58.882839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:58.883117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:58.883388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:58.896582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:58.904822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:58.905827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:41:58.907026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 131/282] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:58.915688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:59.040821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44045e70-a0b8-4314-a2fa-a98846763ff1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:59.076206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:59.081907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:59.083026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:59.083680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:59.084344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:59.085108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:41:59.085447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:59.086843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:59.096073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 152/283] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:59.102394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:59.215624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642d45d-767e-403e-98cf-66f8b029606c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:59.266937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:59.267841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:59.268125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:59.268405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] PATH_INFO: `/v3/users/ddf3a5e2e3604552b62f84e0cfc5dda7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:59.269737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf3a5e2e3604552b62f84e0cfc5dda7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:59.271201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:59.272959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 132/284] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:41:59 2022] GET /identity/v3/users/ddf3a5e2e3604552b62f84e0cfc5dda7 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:59.278673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:59.414825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca9075-9de3-4ce1-9d64-84e5cf661e51 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:59.449257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3ee93-5e3f-496d-ac3d-da7b9ac9b02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:59.450136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3ee93-5e3f-496d-ac3d-da7b9ac9b02e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:59.450420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3ee93-5e3f-496d-ac3d-da7b9ac9b02e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:59.450695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3ee93-5e3f-496d-ac3d-da7b9ac9b02e None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:59.451900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3ee93-5e3f-496d-ac3d-da7b9ac9b02e None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:59.453377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3ee93-5e3f-496d-ac3d-da7b9ac9b02e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:59.455379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 153/285] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:41:59 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:41:59.461581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:59.492303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b19be-693a-4b29-bc13-39bbae95962f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:59.492607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b19be-693a-4b29-bc13-39bbae95962f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:59.492860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b19be-693a-4b29-bc13-39bbae95962f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:59.494415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 154/286] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:41:59.496716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:59.502479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009150d-c5fb-4855-818f-a4e9c630c80a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:59.502767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009150d-c5fb-4855-818f-a4e9c630c80a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:59.503045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009150d-c5fb-4855-818f-a4e9c630c80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:59.525042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0009150d-c5fb-4855-818f-a4e9c630c80a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:41:59.526238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0009150d-c5fb-4855-818f-a4e9c630c80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:41:59.583994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:59.591407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae731bd-e24d-41ce-9005-731f2b17e3c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:59.601177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 155/287] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:41:59.626539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009150d-c5fb-4855-818f-a4e9c630c80a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:59.642059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:59.644479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:59.644841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:59.645863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/ddf3a5e2e3604552b62f84e0cfc5dda7/roles/3208335c205449e6998150af5b78eaa2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:59.651923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8a47ac1834033aef7a1d6151bd94e, user_id=ddf3a5e2e3604552b62f84e0cfc5dda7, role_id=3208335c205449e6998150af5b78eaa2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:41:59.653428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:41:59.685355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 133/288] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:41:59 2022] PUT /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/users/ddf3a5e2e3604552b62f84e0cfc5dda7/roles/3208335c205449e6998150af5b78eaa2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:41:59.785748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009150d-c5fb-4855-818f-a4e9c630c80a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:41:59.833427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:41:59.834893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:41:59.835248 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:41:59.835528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:41:59.855350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:41:59.863660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:41:59.864743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:41:59.865957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:41:59.866593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 156/289] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:41:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:41:59.873032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:41:59.967000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:59.991307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185a142-c5db-4e76-9a56-b5d7f90786d3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:00.041272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:00.041965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:00.042252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:00.042528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:00.043352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:00.043600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:00.044976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:00.054871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 134/290] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:00.066347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:00.153432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:00.236781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d990145-4e08-4317-a27c-2378c327c461 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:00.278208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:00.279086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:00.279368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:00.279641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] PATH_INFO: `/v3/users/330dd838535b48f5aac96898aa501b6c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:00.281967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] RBAC: Authorizing `identity:get_user(user_id=330dd838535b48f5aac96898aa501b6c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:00.283441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:00.285279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 157/291] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:42:00 2022] GET /identity/v3/users/330dd838535b48f5aac96898aa501b6c => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:00.291082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:00.418400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b58a03-5097-4309-986d-a18809b28574 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:00.462093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:00.462812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:00.463030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:00.463272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:00.481632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:00.555406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:00.556812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:00.692354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 135/292] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:00.697818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:00.831061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f752a92-92f9-4478-b66f-54ef43022e39 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:00.865174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:00.882819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf5714-e006-46c1-b6e4-a3727cbbc377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:00.883691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf5714-e006-46c1-b6e4-a3727cbbc377 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:00.883975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf5714-e006-46c1-b6e4-a3727cbbc377 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:00.884247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf5714-e006-46c1-b6e4-a3727cbbc377 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:00.889865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf5714-e006-46c1-b6e4-a3727cbbc377 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:00.891347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf5714-e006-46c1-b6e4-a3727cbbc377 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:00.893368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 158/293] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:00 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:00.899562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:00.963346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:01.043892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20bae-cd73-48b3-868a-a00a9384e68b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:01.077788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e62e6-fcd0-4673-a463-562d35678d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:01.078754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e62e6-fcd0-4673-a463-562d35678d50 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.079181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e62e6-fcd0-4673-a463-562d35678d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.079634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e62e6-fcd0-4673-a463-562d35678d50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.080951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e62e6-fcd0-4673-a463-562d35678d50 None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=330dd838535b48f5aac96898aa501b6c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:01.081434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e62e6-fcd0-4673-a463-562d35678d50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:01.082899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e62e6-fcd0-4673-a463-562d35678d50 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:01.090593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 136/294] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:00 2022] GET /identity/v3/role_assignments?user.id=330dd838535b48f5aac96898aa501b6c&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:01.478275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23449eb-6c36-4b9d-a533-1aedbb8d5ca7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.478783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23449eb-6c36-4b9d-a533-1aedbb8d5ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.478857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23449eb-6c36-4b9d-a533-1aedbb8d5ca7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.480420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 159/295] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:01.487078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ceb17-c8ed-4d4d-8a41-9bc1051f29f4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.487529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ceb17-c8ed-4d4d-8a41-9bc1051f29f4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.487712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ceb17-c8ed-4d4d-8a41-9bc1051f29f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.509274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-844ceb17-c8ed-4d4d-8a41-9bc1051f29f4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:01.510454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-844ceb17-c8ed-4d4d-8a41-9bc1051f29f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:01.597836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 137/296] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:01.623317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23449eb-6c36-4b9d-a533-1aedbb8d5ca7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:01.748708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23449eb-6c36-4b9d-a533-1aedbb8d5ca7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:01.785623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee3b39-837b-422a-a9f7-011de7a99335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:01.786428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee3b39-837b-422a-a9f7-011de7a99335 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.786710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee3b39-837b-422a-a9f7-011de7a99335 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.786985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee3b39-837b-422a-a9f7-011de7a99335 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.798759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee3b39-837b-422a-a9f7-011de7a99335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9cee3b39-837b-422a-a9f7-011de7a99335 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:01.806636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:01.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:01.808351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 160/297] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:01.813800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ceb17-c8ed-4d4d-8a41-9bc1051f29f4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:01.857797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:01.868432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e23f1-3bf2-4841-8ee6-009efd96a058 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.869051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e23f1-3bf2-4841-8ee6-009efd96a058 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.869555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e23f1-3bf2-4841-8ee6-009efd96a058 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.871380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 161/298] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:01.877085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb5174-4a9e-409a-8f62-d86439b888f5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.877685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb5174-4a9e-409a-8f62-d86439b888f5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.878180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb5174-4a9e-409a-8f62-d86439b888f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.895382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fffb5174-4a9e-409a-8f62-d86439b888f5 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:01.897418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fffb5174-4a9e-409a-8f62-d86439b888f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:01.930043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ceb17-c8ed-4d4d-8a41-9bc1051f29f4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:01.945440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:01.967529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25405c1d-690e-4ad4-9c36-8392d77e0b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:01.968220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25405c1d-690e-4ad4-9c36-8392d77e0b6a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.968497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25405c1d-690e-4ad4-9c36-8392d77e0b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.968938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25405c1d-690e-4ad4-9c36-8392d77e0b6a None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.969877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25405c1d-690e-4ad4-9c36-8392d77e0b6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:01.970196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25405c1d-690e-4ad4-9c36-8392d77e0b6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:01.970879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 162/299] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:01.981285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25405c1d-690e-4ad4-9c36-8392d77e0b6a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:01.981451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d77d-d02f-4d2e-a5aa-439f3cfd1707 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.981451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d77d-d02f-4d2e-a5aa-439f3cfd1707 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.981451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d77d-d02f-4d2e-a5aa-439f3cfd1707 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.981784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 138/300] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:01.981870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 163/301] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:01.986935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13377-000e-4d78-9694-74bb6003eda0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:01.987229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13377-000e-4d78-9694-74bb6003eda0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:01.987512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13377-000e-4d78-9694-74bb6003eda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:01.989729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d77d-d02f-4d2e-a5aa-439f3cfd1707 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.007888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-23a13377-000e-4d78-9694-74bb6003eda0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:02.009068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-23a13377-000e-4d78-9694-74bb6003eda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:02.077005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 139/302] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:02.084245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a13377-000e-4d78-9694-74bb6003eda0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.109510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:02.121693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d77d-d02f-4d2e-a5aa-439f3cfd1707 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.159700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.160582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.160864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.161133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] PATH_INFO: `/v3/users/5e25a5b51d424d65b8467cf141191aff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.162887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] RBAC: Authorizing `identity:get_user(user_id=5e25a5b51d424d65b8467cf141191aff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:02.164375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:02.166258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 164/303] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:42:01 2022] GET /identity/v3/users/5e25a5b51d424d65b8467cf141191aff => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:02.171243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.191330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a13377-000e-4d78-9694-74bb6003eda0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.221996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:02.228571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.229405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.229691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.229963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.231143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:02.232586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:02.238784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 140/304] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:02.238784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.286706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a69b87-4fa7-43d9-b1a2-7b7f7955a471 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.320206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.320989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.321298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.321572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.331547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:02.334059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:02.339817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:02.340905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 165/305] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:02.342073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.345036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5d911-6a9b-4544-b4c2-65ffe5e0e085 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.379405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424c8d2-071a-44cd-8275-25b58783ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.380273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c8d2-071a-44cd-8275-25b58783ce25 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.380560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c8d2-071a-44cd-8275-25b58783ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.380838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424c8d2-071a-44cd-8275-25b58783ce25 None admin] PATH_INFO: `/v3/system/users/5e25a5b51d424d65b8467cf141191aff/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.382388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424c8d2-071a-44cd-8275-25b58783ce25 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5e25a5b51d424d65b8467cf141191aff, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:02.383773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424c8d2-071a-44cd-8275-25b58783ce25 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:02.392531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 141/306] 10.222.0.91 () {58 vars in 1334 bytes} [Thu May 26 13:42:02 2022] PUT /identity/v3/system/users/5e25a5b51d424d65b8467cf141191aff/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:02.393823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4f24f-a12a-46d8-aab9-55c760869c2f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.394108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4f24f-a12a-46d8-aab9-55c760869c2f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.394379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4f24f-a12a-46d8-aab9-55c760869c2f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.395651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 142/307] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:02.396363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4f24f-a12a-46d8-aab9-55c760869c2f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.465810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd1b9e-550d-4f1c-b830-abaed0fc2d4d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.473942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:02.506090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d8791-3409-47b5-9138-23c8fa07f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.507532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8791-3409-47b5-9138-23c8fa07f35a None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.507959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8791-3409-47b5-9138-23c8fa07f35a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.508376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8791-3409-47b5-9138-23c8fa07f35a None admin] PATH_INFO: `/v3/groups` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.509244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d8791-3409-47b5-9138-23c8fa07f35a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:02.510989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d8791-3409-47b5-9138-23c8fa07f35a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:02.513054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4f24f-a12a-46d8-aab9-55c760869c2f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.538266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 166/308] 10.222.0.91 () {60 vars in 1134 bytes} [Thu May 26 13:42:02 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:02.539535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334cdb2-b861-4e58-8976-c31e5741510c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.539839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334cdb2-b861-4e58-8976-c31e5741510c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.540188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334cdb2-b861-4e58-8976-c31e5741510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.562093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.563375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.564192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.564752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.565836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:02.566360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:02.568144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9334cdb2-b861-4e58-8976-c31e5741510c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:02.568457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:02.569555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9334cdb2-b861-4e58-8976-c31e5741510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:02.581799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 143/309] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:02.588271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.620950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:02.661441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 167/310] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:02.692592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334cdb2-b861-4e58-8976-c31e5741510c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.727483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1a50e-9058-4b1a-859e-81b082b42c9b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.776215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.777104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.777422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.777700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] PATH_INFO: `/v3/users/e42ba07e5e914fc9a4652b7108ea930e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.794966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] RBAC: Authorizing `identity:get_user(user_id=e42ba07e5e914fc9a4652b7108ea930e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:02.796352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:02.798130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 144/311] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:42:02 2022] GET /identity/v3/users/e42ba07e5e914fc9a4652b7108ea930e => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:02.804975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.823389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:02.827512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334cdb2-b861-4e58-8976-c31e5741510c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.864190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.865478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.866502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.867070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.878746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:02.887568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:02.888484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:02.889365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:02.890829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:02.893124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 168/312] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:02.899645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:02.915343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0768c01-7cbf-4966-9fa6-e4bae1c3db37 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:02.958798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:02.960199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:02.960639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:02.961341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:02.962524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] RBAC: Adding query filter params (role.id=3208335c205449e6998150af5b78eaa2, user.id=e42ba07e5e914fc9a4652b7108ea930e, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:02.962989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:02.965179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:02.972761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 145/313] 10.222.0.91 () {56 vars in 1308 bytes} [Thu May 26 13:42:02 2022] GET /identity/v3/role_assignments?user.id=e42ba07e5e914fc9a4652b7108ea930e&scope.system=all&role.id=3208335c205449e6998150af5b78eaa2 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:03.015842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc119-8e73-41a4-ac06-46494cab1afc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:03.055619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:03.056549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:03.056983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:03.057437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:03.058366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:03.058858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:03.060456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:03.070507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 169/314] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:03.078459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:03.191030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72121b-dc3a-480e-b801-47301b8a4702 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:03.226446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:03.227616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:03.228039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:03.228455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] PATH_INFO: `/v3/users/ddf3a5e2e3604552b62f84e0cfc5dda7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:03.229841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf3a5e2e3604552b62f84e0cfc5dda7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:03.231427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:03.233501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 146/315] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:42:03 2022] GET /identity/v3/users/ddf3a5e2e3604552b62f84e0cfc5dda7 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:03.239246 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:03.264782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:03.349760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500f774-195b-4639-9333-7dd8d86ab920 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:03.384104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:03.384756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:03.385036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:03.385342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:03.455836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:03.457347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:03.478743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:03.560421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 170/316] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:03.566299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:03.581548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:03.691823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceabde-8ca0-4e45-b6f7-87d822e70f06 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:03.728904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfcb61-d26b-4c8f-9f5a-7290a85d790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:03.729802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfcb61-d26b-4c8f-9f5a-7290a85d790a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:03.730087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfcb61-d26b-4c8f-9f5a-7290a85d790a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:03.730364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfcb61-d26b-4c8f-9f5a-7290a85d790a None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:03.731599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfcb61-d26b-4c8f-9f5a-7290a85d790a None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:03.733065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfcb61-d26b-4c8f-9f5a-7290a85d790a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:03.735165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 147/317] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:03 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:03.741341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:03.848812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d524e74-7dcd-4d63-b08d-d92165fd1803 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:03.868523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:03.885480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db7221-5771-4781-8615-32639d2f45fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:03.886490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db7221-5771-4781-8615-32639d2f45fe None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:03.887167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db7221-5771-4781-8615-32639d2f45fe None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:03.887520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db7221-5771-4781-8615-32639d2f45fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:03.888430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db7221-5771-4781-8615-32639d2f45fe None admin] RBAC: Adding query filter params (role.id=3208335c205449e6998150af5b78eaa2, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e, user.id=ddf3a5e2e3604552b62f84e0cfc5dda7) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:03.888812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db7221-5771-4781-8615-32639d2f45fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:03.890356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db7221-5771-4781-8615-32639d2f45fe None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:03.897322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 171/318] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:03 2022] GET /identity/v3/role_assignments?user.id=ddf3a5e2e3604552b62f84e0cfc5dda7&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=3208335c205449e6998150af5b78eaa2 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:04.800796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292189d-2807-4b87-b736-7539299d4bca None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:04.801380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292189d-2807-4b87-b736-7539299d4bca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:04.801380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292189d-2807-4b87-b736-7539299d4bca None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:04.803390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 148/319] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:04.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408f330-a7ed-4247-9e92-5a63e52fbe77 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:04.808945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408f330-a7ed-4247-9e92-5a63e52fbe77 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:04.809130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408f330-a7ed-4247-9e92-5a63e52fbe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:04.826532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e408f330-a7ed-4247-9e92-5a63e52fbe77 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:04.827731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e408f330-a7ed-4247-9e92-5a63e52fbe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:04.878795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45aaa-69ce-425a-9e25-e9b76ae63d83 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:04.879106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45aaa-69ce-425a-9e25-e9b76ae63d83 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:04.879469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45aaa-69ce-425a-9e25-e9b76ae63d83 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:04.881448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 149/320] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:04.886839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3e16-7190-4971-8901-ef028ea833b2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:04.887132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3e16-7190-4971-8901-ef028ea833b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:04.887412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3e16-7190-4971-8901-ef028ea833b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:04.902977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 172/321] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:04.906244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e7a3e16-7190-4971-8901-ef028ea833b2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:04.908063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e7a3e16-7190-4971-8901-ef028ea833b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:04.926439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408f330-a7ed-4247-9e92-5a63e52fbe77 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:04.983559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 150/322] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:05.006866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a3e16-7190-4971-8901-ef028ea833b2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.017766 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:05.033403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408f330-a7ed-4247-9e92-5a63e52fbe77 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.049599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:05.074453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.076302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.076592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.076880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.095156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:05.107437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:05.108547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:05.115975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 173/323] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:05.117460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.131448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a3e16-7190-4971-8901-ef028ea833b2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.167737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.168546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.168837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.169113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.170342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:05.171828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:05.173909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 151/324] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:05.180011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.229842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca67bec-4998-400a-ab31-2a1c720f9a89 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.262288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:05.266483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02e3ea-c498-4db1-9532-3f30de0c3062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.267157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02e3ea-c498-4db1-9532-3f30de0c3062 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.267600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02e3ea-c498-4db1-9532-3f30de0c3062 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.267882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02e3ea-c498-4db1-9532-3f30de0c3062 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.269045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02e3ea-c498-4db1-9532-3f30de0c3062 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:05.269870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02e3ea-c498-4db1-9532-3f30de0c3062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:05.271302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02e3ea-c498-4db1-9532-3f30de0c3062 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:05.279848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 174/325] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:05.282574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b660e-bf39-4f59-9484-24a672066041 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.283119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b660e-bf39-4f59-9484-24a672066041 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.283297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b660e-bf39-4f59-9484-24a672066041 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.284573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 175/326] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:05.285316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b660e-bf39-4f59-9484-24a672066041 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.288583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8db54-1993-4bd3-8e57-55e35e9470fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.320568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:05.323521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b18f4f-2ced-49f3-837f-fd6e5cb13816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.324177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18f4f-2ced-49f3-837f-fd6e5cb13816 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.324506 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18f4f-2ced-49f3-837f-fd6e5cb13816 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.324727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18f4f-2ced-49f3-837f-fd6e5cb13816 None admin] PATH_INFO: `/v3/groups` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.325444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b18f4f-2ced-49f3-837f-fd6e5cb13816 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:05.326881 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b18f4f-2ced-49f3-837f-fd6e5cb13816 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:05.342000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 152/327] 10.222.0.91 () {60 vars in 1134 bytes} [Thu May 26 13:42:05 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:05.343308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6600-9c72-4dc9-ac6f-8fe2899b5e18 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.343608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6600-9c72-4dc9-ac6f-8fe2899b5e18 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.343890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6600-9c72-4dc9-ac6f-8fe2899b5e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.362271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6dfb6600-9c72-4dc9-ac6f-8fe2899b5e18 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:05.363496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6dfb6600-9c72-4dc9-ac6f-8fe2899b5e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:05.406782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b660e-bf39-4f59-9484-24a672066041 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.463048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.463933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.464407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 153/328] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:05.469424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.469790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] PATH_INFO: `/v3/users/5e25a5b51d424d65b8467cf141191aff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.471230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] RBAC: Authorizing `identity:get_user(user_id=5e25a5b51d424d65b8467cf141191aff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:05.472827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:05.475316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 176/329] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:42:05 2022] GET /identity/v3/users/5e25a5b51d424d65b8467cf141191aff => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:05.481191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6600-9c72-4dc9-ac6f-8fe2899b5e18 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.486912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.562962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:05.603599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e9396-86ff-4f5c-a61a-21512c2067b4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.604597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6600-9c72-4dc9-ac6f-8fe2899b5e18 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.613780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:05.648405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.648879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.649247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.649535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.649697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.649801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.649959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.650198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.651035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] RBAC: Adding query filter params (role.id=8719b7bceaea49a48ff010ff8d107832, user.id=5e25a5b51d424d65b8467cf141191aff, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:05.651342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:05.653390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:05.660877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:05.668468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 154/330] 10.222.0.91 () {56 vars in 1308 bytes} [Thu May 26 13:42:05 2022] GET /identity/v3/role_assignments?user.id=5e25a5b51d424d65b8467cf141191aff&scope.system=all&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:05.674540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:05.675552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:05.676367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 177/331] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:05.682258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.803889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b993c-d95a-499d-a02b-a8a0db3054c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:05.834933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:05.847882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:05.848566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:05.848848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:05.849121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:05.849955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:05.850259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:05.851798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:05.861612 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 155/332] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:05.867864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:05.980144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164bea5-650c-488e-b250-b3cd15ba3a14 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:06.014862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22b857-57a0-4abc-87de-8ef5e74ac322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:06.015878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b857-57a0-4abc-87de-8ef5e74ac322 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:06.016207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b857-57a0-4abc-87de-8ef5e74ac322 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:06.016485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b857-57a0-4abc-87de-8ef5e74ac322 None admin] PATH_INFO: `/v3/users/e42ba07e5e914fc9a4652b7108ea930e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:06.017743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22b857-57a0-4abc-87de-8ef5e74ac322 None admin] RBAC: Authorizing `identity:get_user(user_id=e42ba07e5e914fc9a4652b7108ea930e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:06.019223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22b857-57a0-4abc-87de-8ef5e74ac322 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:06.021179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 178/333] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:42:05 2022] GET /identity/v3/users/e42ba07e5e914fc9a4652b7108ea930e => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:06.027273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:06.143435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66298ff2-80e0-47da-9f20-f70b73058e58 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:06.184997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73ee7c-1a40-45e3-963c-9b0de9a33025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:06.185968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ee7c-1a40-45e3-963c-9b0de9a33025 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:06.186244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ee7c-1a40-45e3-963c-9b0de9a33025 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:06.186518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ee7c-1a40-45e3-963c-9b0de9a33025 None admin] PATH_INFO: `/v3/system/users/e42ba07e5e914fc9a4652b7108ea930e/roles/3208335c205449e6998150af5b78eaa2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:06.188017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73ee7c-1a40-45e3-963c-9b0de9a33025 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e42ba07e5e914fc9a4652b7108ea930e, role_id=3208335c205449e6998150af5b78eaa2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:06.194806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73ee7c-1a40-45e3-963c-9b0de9a33025 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:06.206620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 156/334] 10.222.0.91 () {58 vars in 1334 bytes} [Thu May 26 13:42:06 2022] PUT /identity/v3/system/users/e42ba07e5e914fc9a4652b7108ea930e/roles/3208335c205449e6998150af5b78eaa2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:07.784886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099659-42b6-4d14-8a1f-3e089a99cf9d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:07.785427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099659-42b6-4d14-8a1f-3e089a99cf9d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:07.785503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099659-42b6-4d14-8a1f-3e089a99cf9d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:07.787719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 179/335] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:07.794136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5fb09-016b-46a2-b9f9-7e41eb254509 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:07.794633 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5fb09-016b-46a2-b9f9-7e41eb254509 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:07.794712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5fb09-016b-46a2-b9f9-7e41eb254509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:07.812447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ddc5fb09-016b-46a2-b9f9-7e41eb254509 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:07.813687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ddc5fb09-016b-46a2-b9f9-7e41eb254509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:07.875632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:07.900578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 157/336] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:07.924787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09099659-42b6-4d14-8a1f-3e089a99cf9d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:07.958226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:08.047837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09099659-42b6-4d14-8a1f-3e089a99cf9d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:08.091749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:08.092556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:08.092839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:08.093115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:08.101915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45755-98a4-498e-a389-e5d075d43b7f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:08.102194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45755-98a4-498e-a389-e5d075d43b7f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:08.102476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45755-98a4-498e-a389-e5d075d43b7f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:08.103850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 158/337] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:08.110607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:08.114577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb15c80-0f0a-4cc0-b772-35566b543309 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:08.114875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb15c80-0f0a-4cc0-b772-35566b543309 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:08.115184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb15c80-0f0a-4cc0-b772-35566b543309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:08.119185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:08.120075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:08.120910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:08.125134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 180/338] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:08.126119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:08.162754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eeb15c80-0f0a-4cc0-b772-35566b543309 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:08.163964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eeb15c80-0f0a-4cc0-b772-35566b543309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:08.240896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:08.249952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6c05a-dd00-43f9-be5c-e14c2f0b95b2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:08.293086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:08.295280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:08.295734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:08.296181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:08.297586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:08.298501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:08.303484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:08.313245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 181/339] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:08.319054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:08.342744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 159/340] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:08.370533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb15c80-0f0a-4cc0-b772-35566b543309 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:08.513367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db38a46-e144-418f-a8d4-f3ee8c7ff20f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:08.563059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:08.563650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:08.563938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:08.564222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:08.567795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb15c80-0f0a-4cc0-b772-35566b543309 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:08.593529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:08.613860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:08.650559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:08.651397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:08.651683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:08.651956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:08.671157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:08.684041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:08.685007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:08.685848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:08.686494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:08.686578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 160/341] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:42:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:08.689656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:08.694165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:08.856215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 182/342] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:42:08.862028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:08.867174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91594dd5-cded-4c82-8d68-61906d8aaae1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:08.922444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:08.923144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:08.923682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:08.923961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:08.924725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:08.925036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:08.926472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:08.941310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 161/343] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:08.949609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:08.950181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:09.000329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95871886-52f4-4bb9-b575-72ffbea266e7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:09.000978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:09.078691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:09.079582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.079873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.080152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.081348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253c31b-cb3a-47b8-8ff9-17397f4db9b4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:09.085979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:09.087468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:09.089443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 183/344] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:08 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:09.095849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:09.120174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:09.120758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.121041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.121350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.200384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:09.201939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:09.237400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:09.320814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 162/345] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:42:09.321699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f29195-8659-4ff7-9579-a19b54f70e45 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:09.326181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:09.358714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:09.385989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:09.386783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.387069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.393798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.406023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:09.407651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:09.409954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 184/346] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:09 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:09.415563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:09.445902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876a7c5-7e3e-4b2c-8cab-141113865fdf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:09.482181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:09.488199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:09.489242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.489719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.490069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.491637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:09.493263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:09.495700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 163/347] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:09 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:09.502396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:09.563738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ce7b2-d08a-4a3f-931a-96b09de4bf1c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:09.615833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e96e8a-34d9-48a8-b453-51f5f8c7ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:09.616516 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e96e8a-34d9-48a8-b453-51f5f8c7ec98 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.616806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e96e8a-34d9-48a8-b453-51f5f8c7ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.617085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e96e8a-34d9-48a8-b453-51f5f8c7ec98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.618064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e96e8a-34d9-48a8-b453-51f5f8c7ec98 None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=7cc88b5db0244f738c4e498081179b91) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:09.618385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e96e8a-34d9-48a8-b453-51f5f8c7ec98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:09.619951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e96e8a-34d9-48a8-b453-51f5f8c7ec98 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:09.621462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8873bec-660a-4bfb-bac4-618c907746ca None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:09.633318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 185/348] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:09 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:09.639952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ebe8-e83f-4c54-a09f-b479f5f58212 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.640228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ebe8-e83f-4c54-a09f-b479f5f58212 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.640432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ebe8-e83f-4c54-a09f-b479f5f58212 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.641854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 186/349] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:09.648267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04266148-f18d-4c6b-badf-f2bad375c706 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.648559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04266148-f18d-4c6b-badf-f2bad375c706 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.648841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04266148-f18d-4c6b-badf-f2bad375c706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.663761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:09.664636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.664920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.665231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.666421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:09.667974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:09.669955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 164/350] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:09 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:09.676445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:09.682504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-04266148-f18d-4c6b-badf-f2bad375c706 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:09.683761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-04266148-f18d-4c6b-badf-f2bad375c706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:09.752581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:09.754449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:09.792029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aaa61-f30c-44a4-bb6f-b5b5acbb2dfc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:09.803758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 187/351] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:09.830083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd8883-7557-45ad-a95b-94cb913329dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:09.830758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8883-7557-45ad-a95b-94cb913329dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.831046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8883-7557-45ad-a95b-94cb913329dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.831327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd8883-7557-45ad-a95b-94cb913329dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.832252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8883-7557-45ad-a95b-94cb913329dd None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=7cc88b5db0244f738c4e498081179b91) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:09.832571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8883-7557-45ad-a95b-94cb913329dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:09.834652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd8883-7557-45ad-a95b-94cb913329dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:09.841567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 165/352] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:09 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=33a3e99f754a4bc69e665330de985b76 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:09.843361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04266148-f18d-4c6b-badf-f2bad375c706 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:09.991148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93883-2aed-4162-bc1f-3754d76f66e9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:09.991875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93883-2aed-4162-bc1f-3754d76f66e9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:09.991875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93883-2aed-4162-bc1f-3754d76f66e9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:09.996595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 166/353] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:10.001577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04266148-f18d-4c6b-badf-f2bad375c706 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.020710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d2d4-5d8b-4f2b-93d3-ac17ae209e7d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.021035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d2d4-5d8b-4f2b-93d3-ac17ae209e7d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.021303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d2d4-5d8b-4f2b-93d3-ac17ae209e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.058326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-74b8d2d4-5d8b-4f2b-93d3-ac17ae209e7d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:10.059549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74b8d2d4-5d8b-4f2b-93d3-ac17ae209e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:10.062853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.063786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.064073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.064352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.094326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:10.105497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:10.107774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:42:10.109816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:10.111997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 188/354] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:10.113234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.167252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 167/355] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:10.168692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d157-b44b-4e02-8ba4-8e78bbb4cbf2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.169037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d157-b44b-4e02-8ba4-8e78bbb4cbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.169372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d157-b44b-4e02-8ba4-8e78bbb4cbf2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.170800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 168/356] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:10.182407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940cfe5-cb59-4160-8f34-5e752fe86879 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.182797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940cfe5-cb59-4160-8f34-5e752fe86879 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.183152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940cfe5-cb59-4160-8f34-5e752fe86879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.186935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:10.211158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e940cfe5-cb59-4160-8f34-5e752fe86879 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:10.212357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e940cfe5-cb59-4160-8f34-5e752fe86879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:10.282751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:10.291823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0d366-a40e-4a6e-a26a-e6df500ae730 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.296102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 169/357] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:10.296337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940cfe5-cb59-4160-8f34-5e752fe86879 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.336296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.337941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.338676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.339520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.340810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:10.341708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:10.343738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:10.354219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 189/358] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:10.355330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.411994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940cfe5-cb59-4160-8f34-5e752fe86879 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.450405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.451281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.451640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.451993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.464364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:10.472946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:10.473995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:10.474862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:10.479233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 170/359] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:10.480323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.490275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:10.494624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fced8-3d9e-4085-b400-c1b16c95f9e6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.541502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.542493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.542788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.543165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.551306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:10.558934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:10.568689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:10.569696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:10.570531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:10.573314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 190/360] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:10.574474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.610982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78618a06-7a3b-4119-b4ce-ecb98b632a30 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.652321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.653390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.653615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.653899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] PATH_INFO: `/v3/users/e42ba07e5e914fc9a4652b7108ea930e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.655470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] RBAC: Authorizing `identity:get_user(user_id=e42ba07e5e914fc9a4652b7108ea930e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:10.656693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:10.663274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 171/361] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/users/e42ba07e5e914fc9a4652b7108ea930e => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:10.664283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.691055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7af01a-2357-40ce-816d-57ef657db62a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.729602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.730820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.731353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.731835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.732916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:10.733572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:10.735647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:10.748026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 191/362] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:10.749229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.777923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:10.783232 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80119ed-520d-4d5e-a6c0-7928833ee332 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.808375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:10.828246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.828972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.829729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.829729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.830423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:10.830738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:10.832165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:10.844099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 172/363] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:10.845101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.875937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381a1f7-6c43-454c-af46-75f783071058 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:10.920647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:10.922128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:10.922714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:10.923479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:10.924657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] RBAC: Adding query filter params (role.id=3208335c205449e6998150af5b78eaa2, user.id=e42ba07e5e914fc9a4652b7108ea930e, scope.system=all) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:10.925223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:10.926974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:10.941648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 192/364] 10.222.0.91 () {56 vars in 1308 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/role_assignments?user.id=e42ba07e5e914fc9a4652b7108ea930e&scope.system=all&role.id=3208335c205449e6998150af5b78eaa2 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:10.942803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:10.976932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ca32f-460a-4c4a-bfa5-d5c563f526e6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:11.021023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429039c6-6593-4bb3-9b7c-3bc1805fc0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:11.021880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429039c6-6593-4bb3-9b7c-3bc1805fc0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.022092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429039c6-6593-4bb3-9b7c-3bc1805fc0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.022372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429039c6-6593-4bb3-9b7c-3bc1805fc0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.054076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.077960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2af507-91e1-43df-9093-532ea5d48806 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:11.084274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.106169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429039c6-6593-4bb3-9b7c-3bc1805fc0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:11.107818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429039c6-6593-4bb3-9b7c-3bc1805fc0d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:11.120300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf9d2a-9d7e-4df3-a4c3-fe6ae5fefe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:11.121574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9d2a-9d7e-4df3-a4c3-fe6ae5fefe0b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.122085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9d2a-9d7e-4df3-a4c3-fe6ae5fefe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.122567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9d2a-9d7e-4df3-a4c3-fe6ae5fefe0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.227608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf9d2a-9d7e-4df3-a4c3-fe6ae5fefe0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:11.229684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf9d2a-9d7e-4df3-a4c3-fe6ae5fefe0b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:11.237412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 173/365] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:11.239078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870a19c-ff3b-446a-a4df-9a303b51b95b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.239378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870a19c-ff3b-446a-a4df-9a303b51b95b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.239663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870a19c-ff3b-446a-a4df-9a303b51b95b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.241863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 174/366] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:11.243282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870a19c-ff3b-446a-a4df-9a303b51b95b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:11.342504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.356116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.377719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 193/367] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:11.379152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af3d0-e23c-478a-8369-8c4f0ba806e4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.379441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af3d0-e23c-478a-8369-8c4f0ba806e4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.379731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af3d0-e23c-478a-8369-8c4f0ba806e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.388791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870a19c-ff3b-446a-a4df-9a303b51b95b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:11.401777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6e1af3d0-e23c-478a-8369-8c4f0ba806e4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:11.403410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6e1af3d0-e23c-478a-8369-8c4f0ba806e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:11.433235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:11.434158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.434447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.434724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.436408 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:11.438183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:11.444815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 175/368] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:11 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:11.444815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:11.487540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 194/369] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:11.489064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1af3d0-e23c-478a-8369-8c4f0ba806e4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:11.573193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a93181-7873-48f3-b85d-2cc272189be8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:11.603846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1af3d0-e23c-478a-8369-8c4f0ba806e4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:11.617219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:11.617819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.618101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.618386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.618665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.620338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:11.622151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:11.624523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 176/370] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:11 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:11.625383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:11.649188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.650364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:11.651527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.652100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.653669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.655321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:11.658006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:11.665033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 195/371] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:11 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:11.666430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:11.757295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eab5f3-f9f1-410b-8231-cde4c0d42a15 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:11.796388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:11.797585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.798073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.798823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.803620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759ed3f-677f-4233-a226-e0f8d915b53f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:11.813540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:11.822066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:11.822981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:11.823796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:11.826261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 177/372] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:11.827502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:11.843974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:11.845110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:11.845421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:11.845696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:11.847681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:11.850051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:11.852426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 196/373] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:11 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:11.853368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:11.926043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.932065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:11.959789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f33119-6435-4f48-aa6c-b34e1fc0cd48 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:12.000007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:12.000361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d74acf-d1c2-481f-b72c-2b9fdbd4f041 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:12.000675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:12.000960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:12.001268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:12.002120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:12.002433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:12.004197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:12.011877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 178/374] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:11 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:12.013175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:12.043857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:12.044598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:12.044887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:12.045178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:12.045958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:12.046267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:12.047720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:12.060101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 197/375] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:12.065923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:12.152734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecbc39-cfe2-4b76-b5f9-c1fb23983c0c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:12.197756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9fe12-533a-4451-9fc0-67f8e61bc702 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:12.208687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:12.219672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:12.220341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:12.220625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:12.220902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:12.221799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:12.222111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:12.223574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:12.230708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 179/376] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:12 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=33a3e99f754a4bc69e665330de985b76 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:12.244220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:12.244958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:12.245272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:12.245553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:12.344801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:12.350673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:12.463812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 198/377] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:42:12.469521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:12.479977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:12.591791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac72036-5330-49c1-9271-78a4b4415373 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:12.631439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:12.632318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:12.632597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:12.632871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] PATH_INFO: `/v3/projects/e741295126a84576b8368706b7ffe76d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:12.634176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] RBAC: Authorizing `identity:get_project(project_id=e741295126a84576b8368706b7ffe76d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:12.635629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:12.637650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 180/378] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:12 2022] GET /identity/v3/projects/e741295126a84576b8368706b7ffe76d => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:12.643224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:12.654775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:12.752337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e51141-cd01-430f-b32d-dcc844bda785 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:12.788775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e369d-3864-40b3-9fbf-3c7d3fed562b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:12.789692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e369d-3864-40b3-9fbf-3c7d3fed562b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:12.789979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e369d-3864-40b3-9fbf-3c7d3fed562b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:12.790254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e369d-3864-40b3-9fbf-3c7d3fed562b None admin] PATH_INFO: `/v3/groups/bec73769607c49529a8df1aaa170f62a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:12.796297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e369d-3864-40b3-9fbf-3c7d3fed562b None admin] RBAC: Authorizing `identity:get_group(group_id=bec73769607c49529a8df1aaa170f62a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:12.797901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e369d-3864-40b3-9fbf-3c7d3fed562b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:12.800270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 199/379] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:12 2022] GET /identity/v3/groups/bec73769607c49529a8df1aaa170f62a => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:12.807100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:12.889295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:12.917109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005541db-2886-4593-a8e6-4e8518a44489 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:12.952330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b105205-e898-45d1-ad70-e275dce997de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:12.953003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b105205-e898-45d1-ad70-e275dce997de None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:12.953319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b105205-e898-45d1-ad70-e275dce997de None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:12.953597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b105205-e898-45d1-ad70-e275dce997de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:12.954438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b105205-e898-45d1-ad70-e275dce997de None admin] RBAC: Adding query filter params (group.id=bec73769607c49529a8df1aaa170f62a, role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=e741295126a84576b8368706b7ffe76d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:12.954749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b105205-e898-45d1-ad70-e275dce997de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:12.956193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b105205-e898-45d1-ad70-e275dce997de None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:12.963013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 181/380] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:12 2022] GET /identity/v3/role_assignments?group.id=bec73769607c49529a8df1aaa170f62a&scope.project.id=e741295126a84576b8368706b7ffe76d&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:13.019227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844345fe-124e-4f0a-a472-bbb28e912738 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.019524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844345fe-124e-4f0a-a472-bbb28e912738 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.019704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844345fe-124e-4f0a-a472-bbb28e912738 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.022000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 200/381] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:13.031175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568aad-7cfe-4db2-99cf-24779e38fcf3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.031494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568aad-7cfe-4db2-99cf-24779e38fcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.031734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568aad-7cfe-4db2-99cf-24779e38fcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.052029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-74568aad-7cfe-4db2-99cf-24779e38fcf3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:13.053659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74568aad-7cfe-4db2-99cf-24779e38fcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:13.143494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 182/382] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:13.182485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844345fe-124e-4f0a-a472-bbb28e912738 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:13.284485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:13.329138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844345fe-124e-4f0a-a472-bbb28e912738 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:13.371675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f274ca-2ac6-4f02-bad6-f15f37e605ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:13.372549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f274ca-2ac6-4f02-bad6-f15f37e605ca None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.372834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f274ca-2ac6-4f02-bad6-f15f37e605ca None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.373108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f274ca-2ac6-4f02-bad6-f15f37e605ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.389630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f274ca-2ac6-4f02-bad6-f15f37e605ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-95f274ca-2ac6-4f02-bad6-f15f37e605ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:13.398835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:13.399778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:13.400595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:13.404791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 201/383] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:42:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:13.407165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568aad-7cfe-4db2-99cf-24779e38fcf3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:13.475205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe2661-5410-4df4-b49d-89311d45522f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.475509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe2661-5410-4df4-b49d-89311d45522f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.475769 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe2661-5410-4df4-b49d-89311d45522f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.481065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 202/384] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:42:13.482229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9dd81-6158-4f6f-a3e6-a9e40acf2306 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.482515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9dd81-6158-4f6f-a3e6-a9e40acf2306 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.482790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9dd81-6158-4f6f-a3e6-a9e40acf2306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.496144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:13.503450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-39c9dd81-6158-4f6f-a3e6-a9e40acf2306 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:13.504762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-39c9dd81-6158-4f6f-a3e6-a9e40acf2306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:13.563998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568aad-7cfe-4db2-99cf-24779e38fcf3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:13.620462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 203/385] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:13.630001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:13.631227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.631508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.631779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.632521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:13.632823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:13.634242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:13.647391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 183/386] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:13.652883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9dd81-6158-4f6f-a3e6-a9e40acf2306 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:13.685950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:13.745828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:13.815543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71743f-a172-4469-a3c7-d6d838965417 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:13.831567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9dd81-6158-4f6f-a3e6-a9e40acf2306 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:13.869584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:13.879502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:13.880330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.880678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.881468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.907292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:13.908261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:13.908582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:13.908863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:13.910235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:13.911691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:13.913778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 204/387] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:13 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:13.913925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:13.918793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:13.922221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:13.923578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:13.924589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:13.926646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 184/388] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:13.932461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:14.054739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7d93-5f85-49cf-b031-8a1bbc6ad53d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:14.091591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:14.092218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:14.092498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:14.092770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:14.096573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:14.096573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:14.096573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:14.105326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 185/389] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:14.112141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:14.129857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad8c71-f6bb-4a9b-a246-cad87550858e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:14.182696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:14.226773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:14.238952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a971840-4fe1-4c98-83fb-7309b7e76c58 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:14.258404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:14.259645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:14.260135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:14.260614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:14.262091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:14.263759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:14.266028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 205/390] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:13 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:14.271268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:14.275532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:14.276677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:14.276961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:14.277261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:14.279397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:14.280863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:14.283232 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 186/391] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:14 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:14.292425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:14.413259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a1f92-a875-4181-b6d3-ab993dfbb0f6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:14.441529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:14.445459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a62c6a-50e9-4464-a566-0a708bd98280 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:14.450330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:14.451567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:14.451997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:14.452705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:14.454095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:14.455735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:14.458065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 187/392] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:14 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:14.466249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:14.502939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f3031-656a-4bb5-af6d-b07b2d9bc2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:14.503848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f3031-656a-4bb5-af6d-b07b2d9bc2eb None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:14.504139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f3031-656a-4bb5-af6d-b07b2d9bc2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:14.504410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f3031-656a-4bb5-af6d-b07b2d9bc2eb None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:14.506749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f3031-656a-4bb5-af6d-b07b2d9bc2eb None admin] RBAC: Authorizing `identity:create_grant(project_id=7cc88b5db0244f738c4e498081179b91, group_id=f0adfa3c32634b93b51fb7738dc7a9e8, role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:14.508200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f3031-656a-4bb5-af6d-b07b2d9bc2eb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:14.545754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 206/393] 10.222.0.91 () {58 vars in 1442 bytes} [Thu May 26 13:42:14 2022] PUT /identity/v3/projects/7cc88b5db0244f738c4e498081179b91/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/33a3e99f754a4bc69e665330de985b76 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:42:14.585071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8759a4-2801-4b0f-bdc7-47d9d2464dab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:14.625097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26833e33-b37c-475a-a162-7e501999ad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:14.626340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26833e33-b37c-475a-a162-7e501999ad77 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:14.626809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26833e33-b37c-475a-a162-7e501999ad77 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:14.627236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26833e33-b37c-475a-a162-7e501999ad77 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:14.629361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26833e33-b37c-475a-a162-7e501999ad77 None admin] RBAC: Authorizing `identity:create_grant(project_id=7cc88b5db0244f738c4e498081179b91, group_id=f0adfa3c32634b93b51fb7738dc7a9e8, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:14.630997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26833e33-b37c-475a-a162-7e501999ad77 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:14.652852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 188/394] 10.222.0.91 () {58 vars in 1442 bytes} [Thu May 26 13:42:14 2022] PUT /identity/v3/projects/7cc88b5db0244f738c4e498081179b91/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:15.339686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f47c-dc68-47bf-9531-50bee885f137 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:15.340347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f47c-dc68-47bf-9531-50bee885f137 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:15.340347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f47c-dc68-47bf-9531-50bee885f137 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:15.341909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 207/395] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:15.347842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235b4d-9295-4027-a0f7-f7bf10d111fc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:15.348098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235b4d-9295-4027-a0f7-f7bf10d111fc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:15.348361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235b4d-9295-4027-a0f7-f7bf10d111fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:15.374379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9b235b4d-9295-4027-a0f7-f7bf10d111fc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:15.375655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9b235b4d-9295-4027-a0f7-f7bf10d111fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:15.435670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:15.477789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 189/396] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:15.513049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400f47c-dc68-47bf-9531-50bee885f137 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:15.516449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f68c1-c749-4604-afa7-b2c4c6bdc3a7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:15.517234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f68c1-c749-4604-afa7-b2c4c6bdc3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:15.517987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f68c1-c749-4604-afa7-b2c4c6bdc3a7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:15.520077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 190/397] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:42:15.528034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366a9e-8049-4e74-bb65-fc91ee3a5acf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:15.528910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366a9e-8049-4e74-bb65-fc91ee3a5acf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:15.529555 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366a9e-8049-4e74-bb65-fc91ee3a5acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:15.555188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e366a9e-8049-4e74-bb65-fc91ee3a5acf None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:15.557733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e366a9e-8049-4e74-bb65-fc91ee3a5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:15.639858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 191/398] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:15.648830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:15.663038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366a9e-8049-4e74-bb65-fc91ee3a5acf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:15.772410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366a9e-8049-4e74-bb65-fc91ee3a5acf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:15.777895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400f47c-dc68-47bf-9531-50bee885f137 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:15.784217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:15.820634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:15.821976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:15.822508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:15.823006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:15.836141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:15.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:15.846292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:15.847107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:15.850716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 192/399] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:42:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:15.857072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:15.897923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:15.898747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:15.899029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:15.899300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:15.934460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:15.950419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:15.951408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:15.953205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 208/400] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:15.962204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:15.976717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7ccbf-aa30-448b-8420-d455de251262 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.017903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefa21c-2d39-49dd-8412-7260126c6238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:16.019156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefa21c-2d39-49dd-8412-7260126c6238 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.019707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefa21c-2d39-49dd-8412-7260126c6238 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.020241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefa21c-2d39-49dd-8412-7260126c6238 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.021908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefa21c-2d39-49dd-8412-7260126c6238 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:16.022669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefa21c-2d39-49dd-8412-7260126c6238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:16.025234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefa21c-2d39-49dd-8412-7260126c6238 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:16.040892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 193/401] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:16.042744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4098bd-b9fc-44f0-aae7-917c8d9a17ec None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.043242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4098bd-b9fc-44f0-aae7-917c8d9a17ec None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.043526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4098bd-b9fc-44f0-aae7-917c8d9a17ec None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.045097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 194/402] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:16.046047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4098bd-b9fc-44f0-aae7-917c8d9a17ec None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:16.080949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:16.183206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4098bd-b9fc-44f0-aae7-917c8d9a17ec None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.184528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47278c1-85ff-476a-aa78-da00e87cacc8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.203322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:16.231097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358d53c-90d4-4b20-b6ed-bb6a9c10ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:16.231776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d53c-90d4-4b20-b6ed-bb6a9c10ce1a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.232056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d53c-90d4-4b20-b6ed-bb6a9c10ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.232327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d53c-90d4-4b20-b6ed-bb6a9c10ce1a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.233432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358d53c-90d4-4b20-b6ed-bb6a9c10ce1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:16.233750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358d53c-90d4-4b20-b6ed-bb6a9c10ce1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:16.235270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358d53c-90d4-4b20-b6ed-bb6a9c10ce1a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:16.246849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 209/403] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:16.248036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb4e1-92a5-4f7a-9e47-9a362c211527 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.248328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb4e1-92a5-4f7a-9e47-9a362c211527 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.248814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb4e1-92a5-4f7a-9e47-9a362c211527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.249631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:16.251285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.251780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.252316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.254559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:16.258055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:16.262463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 195/404] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:16 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:16.263918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:16.277145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fa9cb4e1-92a5-4f7a-9e47-9a362c211527 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:16.278295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fa9cb4e1-92a5-4f7a-9e47-9a362c211527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:16.355115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 210/405] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:16.356048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb4e1-92a5-4f7a-9e47-9a362c211527 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:16.453070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:16.460169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef306-847b-4a01-9c2e-c54b93feefee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.471733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb4e1-92a5-4f7a-9e47-9a362c211527 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.485937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:16.506973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:16.508191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.508680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.509191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.512152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:16.513281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:16.514434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:16.515564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.515764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 196/406] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:16 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:16.516169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.516582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.516793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:16.522549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:16.524371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:16.526998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 211/407] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:16 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:16.536503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:16.664176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2f738-62fb-4d40-bcba-525fbe30c5c1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.689762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770187e-36a1-48bf-a1aa-6cce482620bb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.706254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:16.707246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.707667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.708044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.722175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:16.730446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:16.731378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:16.732401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 197/408] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:16.736366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:16.737414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:16.738610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:16.739239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:16.739688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:16.743210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8a47ac1834033aef7a1d6151bd94e, group_id=f0adfa3c32634b93b51fb7738dc7a9e8, role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:16.746578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:16.768358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 212/409] 10.222.0.91 () {58 vars in 1442 bytes} [Thu May 26 13:42:16 2022] PUT /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/33a3e99f754a4bc69e665330de985b76 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:16.769619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:16.788707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:16.848715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:16.969252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef11270-0fa3-4d57-bce0-6fa780e23e63 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:16.979890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e77b5b-fac6-4783-8f52-6884a16aea83 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:17.031568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:17.032297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.032618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.032902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.034876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:17.035285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:17.037277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:17.048681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 213/410] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:17.056451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:17.057872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:17.058830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.059190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.059561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.061107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:17.063647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:17.066140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 198/411] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:16 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:17.072486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:17.207066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d4401-722e-4ed5-a5c1-fc0ea8482cb1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:17.209622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:17.241526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:17.247386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458574eb-64c2-4ba5-9b26-385e82ca1a18 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:17.273822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:17.274730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.275016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.275294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.278425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8a47ac1834033aef7a1d6151bd94e, group_id=f0adfa3c32634b93b51fb7738dc7a9e8, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:17.279883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:17.304780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 199/412] 10.222.0.91 () {58 vars in 1442 bytes} [Thu May 26 13:42:17 2022] PUT /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e/groups/f0adfa3c32634b93b51fb7738dc7a9e8/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:17.312772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:17.313753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.314050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.314336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] PATH_INFO: `/v3/groups/bec73769607c49529a8df1aaa170f62a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.315646 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] RBAC: Authorizing `identity:get_group(group_id=bec73769607c49529a8df1aaa170f62a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:17.317351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:17.319910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 214/413] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:17 2022] GET /identity/v3/groups/bec73769607c49529a8df1aaa170f62a => generated 229 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:17.326539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:17.478121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e8ef-fa2d-46ad-b6cd-4a846c6b76d2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:17.528079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616b526-9c60-46f5-846f-971ed708d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:17.528976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616b526-9c60-46f5-846f-971ed708d24f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.529296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616b526-9c60-46f5-846f-971ed708d24f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.529576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616b526-9c60-46f5-846f-971ed708d24f None admin] PATH_INFO: `/v3/projects/e741295126a84576b8368706b7ffe76d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.530963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616b526-9c60-46f5-846f-971ed708d24f None admin] RBAC: Authorizing `identity:get_project(project_id=e741295126a84576b8368706b7ffe76d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:17.532481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616b526-9c60-46f5-846f-971ed708d24f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:17.534791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 200/414] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:17 2022] GET /identity/v3/projects/e741295126a84576b8368706b7ffe76d => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:17.542309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:17.679030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04578b02-d827-46b2-8fcc-75d0fb561f2c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:17.715442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9aeac-ae41-4d98-b157-216c3512e640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:17.716383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9aeac-ae41-4d98-b157-216c3512e640 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.716721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9aeac-ae41-4d98-b157-216c3512e640 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.717005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9aeac-ae41-4d98-b157-216c3512e640 None admin] PATH_INFO: `/v3/projects/e741295126a84576b8368706b7ffe76d/groups/bec73769607c49529a8df1aaa170f62a/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.719043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9aeac-ae41-4d98-b157-216c3512e640 None admin] RBAC: Authorizing `identity:create_grant(project_id=e741295126a84576b8368706b7ffe76d, group_id=bec73769607c49529a8df1aaa170f62a, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:17.720570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9aeac-ae41-4d98-b157-216c3512e640 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:17.722471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a32e4-2ea5-46b8-b8f1-250d789b2158 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.722766 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a32e4-2ea5-46b8-b8f1-250d789b2158 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.723046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a32e4-2ea5-46b8-b8f1-250d789b2158 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.725228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 201/415] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:17.731469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a1faa-d366-4920-a20e-464af0e4bb1a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.731761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a1faa-d366-4920-a20e-464af0e4bb1a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.732029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a1faa-d366-4920-a20e-464af0e4bb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.741497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 215/416] 10.222.0.91 () {58 vars in 1442 bytes} [Thu May 26 13:42:17 2022] PUT /identity/v3/projects/e741295126a84576b8368706b7ffe76d/groups/bec73769607c49529a8df1aaa170f62a/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:17.765502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b15a1faa-d366-4920-a20e-464af0e4bb1a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:17.767298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b15a1faa-d366-4920-a20e-464af0e4bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:17.792739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:17.831913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05595f-b17f-46c1-8af3-c418ac76af60 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.832258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05595f-b17f-46c1-8af3-c418ac76af60 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.832475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05595f-b17f-46c1-8af3-c418ac76af60 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.834321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 216/417] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:17.844250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bd8e3-81b6-4812-bc18-43dfb32ad690 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:17.844250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bd8e3-81b6-4812-bc18-43dfb32ad690 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:17.844250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bd8e3-81b6-4812-bc18-43dfb32ad690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:17.869986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-260bd8e3-81b6-4812-bc18-43dfb32ad690 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:17.871183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-260bd8e3-81b6-4812-bc18-43dfb32ad690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:17.871728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 202/418] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:17.915227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:17.933098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a1faa-d366-4920-a20e-464af0e4bb1a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:17.984285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 217/419] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:18.023857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bd8e3-81b6-4812-bc18-43dfb32ad690 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:18.078241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a1faa-d366-4920-a20e-464af0e4bb1a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:18.143373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bd8e3-81b6-4812-bc18-43dfb32ad690 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:18.156653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:18.157750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:18.158173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:18.158584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:18.188274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:18.194990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:18.195823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:18.196107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:18.196384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:18.207302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:18.208234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:18.209614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:18.216669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 203/420] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:18.221372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:18.226549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:18.243462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:18.244412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:18.245262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:18.246412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 218/421] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:42:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:18.254519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:18.256172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:18.304419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:18.415020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d730f3-bcb4-4df0-9966-21730e18d62a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:18.459832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f094-04a4-4847-a391-602ff8da9293 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:18.499247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:18.500730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:18.501391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:18.501882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:18.502965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:18.503469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:18.505054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:18.517428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:18.518511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:18.518993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 204/422] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:18.519484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:18.519952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:18.520994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:18.522603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:18.523506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:18.525498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:18.543547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 219/423] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:18.549628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:18.641689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:18.666333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d20cf-8639-4325-b3da-dc29eb8605c2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:18.707062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:18.708266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:18.709348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:18.709823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:18.720046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:18.790583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae65f1-8e2e-4f2a-9a04-27ccdec93377 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:18.797068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:18.800100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:18.866116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:18.867971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:18.869509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:18.870578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:18.965055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:18.966863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:18.992845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:19.027023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 220/424] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:19.033020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:19.141828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345634c-2465-431e-822c-0b602d9803f9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:19.180164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:19.187167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:19.188044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.188314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.188582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.189809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:19.191432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:19.194001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 221/425] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:19 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:19.199244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:19.221116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 205/426] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:19.227087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:19.289662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:19.395288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bac0f-6e24-4b54-bedc-d9d4b9cae342 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:19.416151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99edb3f-5ffc-4e6a-8a00-36a233da928a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:19.467281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:19.468316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.468596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.468868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] PATH_INFO: `/v3/projects/7cc88b5db0244f738c4e498081179b91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.470600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] RBAC: Authorizing `identity:get_project(project_id=7cc88b5db0244f738c4e498081179b91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:19.472056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:19.475102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:19.476045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.476326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.476594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.478057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:19.479590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:19.480829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 206/427] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:19 2022] GET /identity/v3/projects/7cc88b5db0244f738c4e498081179b91 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:19.481678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:19.482934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 222/428] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:19 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:19.488124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:19.567529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:19.668401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11675a2c-7e16-4ebd-a1f7-96b01247ac96 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:19.671096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8a1e-e74a-44de-b006-564323da19f8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:19.697558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:19.715919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fd7a1-462b-42c6-9967-51f5fe84d62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:19.716590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fd7a1-462b-42c6-9967-51f5fe84d62c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.716870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fd7a1-462b-42c6-9967-51f5fe84d62c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.717144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fd7a1-462b-42c6-9967-51f5fe84d62c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.718192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fd7a1-462b-42c6-9967-51f5fe84d62c None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=7cc88b5db0244f738c4e498081179b91) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:19.718507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fd7a1-462b-42c6-9967-51f5fe84d62c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:19.720182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fd7a1-462b-42c6-9967-51f5fe84d62c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:19.732279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 223/429] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:19 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=33a3e99f754a4bc69e665330de985b76 => generated 611 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:19.740402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:19.741346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.741624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.741892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.744349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cc22a-a3b3-4740-9b5f-8e52d7facdf6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.744704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cc22a-a3b3-4740-9b5f-8e52d7facdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.744893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cc22a-a3b3-4740-9b5f-8e52d7facdf6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.745781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:19.746687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 224/430] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:42:19.747681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:19.753310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 207/431] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:19 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:19.756272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3b4e6-441d-4008-8122-dca4dc169733 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.756488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:19.757041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3b4e6-441d-4008-8122-dca4dc169733 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.757623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3b4e6-441d-4008-8122-dca4dc169733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.776702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-79f3b4e6-441d-4008-8122-dca4dc169733 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:19.778532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-79f3b4e6-441d-4008-8122-dca4dc169733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:19.857856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 225/432] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:19.882751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3b4e6-441d-4008-8122-dca4dc169733 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:19.920763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cfe13-3eea-417b-b9d5-f5e7c140eb91 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:19.939016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:19.975400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90130c-b38f-472c-b469-817f143b965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:19.976463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90130c-b38f-472c-b469-817f143b965e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.977025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90130c-b38f-472c-b469-817f143b965e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.977338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90130c-b38f-472c-b469-817f143b965e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.978365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90130c-b38f-472c-b469-817f143b965e None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=7cc88b5db0244f738c4e498081179b91) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:19.979036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90130c-b38f-472c-b469-817f143b965e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:19.980968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90130c-b38f-472c-b469-817f143b965e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:19.991522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 208/433] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:19 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=7cc88b5db0244f738c4e498081179b91&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 611 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:19.993044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28066e-0ccf-4745-9094-0d3d9174417c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:19.993374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28066e-0ccf-4745-9094-0d3d9174417c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:19.993660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28066e-0ccf-4745-9094-0d3d9174417c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:19.999594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 209/434] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:20.001470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355ecd0-f3f7-4447-b5f4-c9ff8841735f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.001762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355ecd0-f3f7-4447-b5f4-c9ff8841735f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.002040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355ecd0-f3f7-4447-b5f4-c9ff8841735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.023628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0355ecd0-f3f7-4447-b5f4-c9ff8841735f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:20.025648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0355ecd0-f3f7-4447-b5f4-c9ff8841735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:20.040607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3b4e6-441d-4008-8122-dca4dc169733 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:20.059198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:20.091722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:20.092595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.092878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.093176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.107609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:20.118056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:20.119042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:20.119859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 226/435] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:42:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:20.129327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:20.132972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 210/436] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:20.157374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355ecd0-f3f7-4447-b5f4-c9ff8841735f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:20.266001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e36ca-0153-4f3b-870a-09e306a95097 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:20.298544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355ecd0-f3f7-4447-b5f4-c9ff8841735f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:20.301257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:20.313647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:20.314448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.314740 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.315022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.315781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:20.316095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:20.317562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:20.327900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 227/437] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:20.335307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:20.339951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:20.340754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.341035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.341344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.354209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:20.362636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:20.363545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:20.364390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 211/438] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:20.375215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:20.380729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:20.467022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172588b-7d04-4478-b23e-8969c73392de None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:20.510673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64ab56-3468-41f3-84c4-82eb5d63b1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:20.511693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64ab56-3468-41f3-84c4-82eb5d63b1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.512183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64ab56-3468-41f3-84c4-82eb5d63b1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.512661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64ab56-3468-41f3-84c4-82eb5d63b1e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.513592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cae396-d9c6-4ae5-a5f7-d7a980d8a544 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:20.553527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:20.554268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.554550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.555885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:20.556193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:20.558271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:20.570569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 212/439] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:20.577279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:20.607684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64ab56-3468-41f3-84c4-82eb5d63b1e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:20.609208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64ab56-3468-41f3-84c4-82eb5d63b1e0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:20.613015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:20.644258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:20.706121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dfa2d-9a13-4184-9eb2-85e2bc14b1c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:20.733324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 228/440] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:20.734792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346f66e-b7fd-4bbd-8bae-0f1de42a90c1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.735093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346f66e-b7fd-4bbd-8bae-0f1de42a90c1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.735370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346f66e-b7fd-4bbd-8bae-0f1de42a90c1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.736643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 229/441] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:20.739153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346f66e-b7fd-4bbd-8bae-0f1de42a90c1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:20.744378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:20.745025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.745332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.745703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.814164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:20.816044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:20.844168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:20.844767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346f66e-b7fd-4bbd-8bae-0f1de42a90c1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:20.873105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:20.878971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5dd4a-93e1-4904-9bd4-726cdc048bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:20.879834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5dd4a-93e1-4904-9bd4-726cdc048bfa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.880116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5dd4a-93e1-4904-9bd4-726cdc048bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.880890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5dd4a-93e1-4904-9bd4-726cdc048bfa None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.882122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5dd4a-93e1-4904-9bd4-726cdc048bfa None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:20.883558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5dd4a-93e1-4904-9bd4-726cdc048bfa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:20.889863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 230/442] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:20.889863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9204b-d438-49b9-8853-cb5bba118d7b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:20.889863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9204b-d438-49b9-8853-cb5bba118d7b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:20.889863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9204b-d438-49b9-8853-cb5bba118d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:20.904759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-83f9204b-d438-49b9-8853-cb5bba118d7b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:20.905925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-83f9204b-d438-49b9-8853-cb5bba118d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:20.923319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 213/443] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:42:20.929204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:20.975152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 231/444] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:20.975995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9204b-d438-49b9-8853-cb5bba118d7b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.028206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3b6c7-29b0-4706-b7a8-81db341d455e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.054758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:21.063847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.064705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.064986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.065291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.066478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.067947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.069600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 214/445] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:21.070353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.078224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9204b-d438-49b9-8853-cb5bba118d7b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.108135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:21.114779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.115647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.115934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.116211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] PATH_INFO: `/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.117497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc8a47ac1834033aef7a1d6151bd94e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.118956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.120657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 232/446] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:20 2022] GET /identity/v3/projects/dbc8a47ac1834033aef7a1d6151bd94e => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:21.121550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.170624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159455-0224-4a8a-836b-9aa09650d479 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.204829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.205652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.205972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.206197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.216490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:21.222459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3ff75-3f53-4c2e-b78d-6e5fea9f72a5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:21.224696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:21.225695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:21.226518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 215/447] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:21.227186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.260003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.260651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.260931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.261243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.262070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:21.262376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.263816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.269536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 233/448] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=33a3e99f754a4bc69e665330de985b76 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:21.270317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.290263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:21.343475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f89b5c-3175-49cc-8e70-9b49c586bc60 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.353359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:21.382375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.383529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.383815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.384098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] PATH_INFO: `/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.385415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] RBAC: Authorizing `identity:get_group(group_id=f0adfa3c32634b93b51fb7738dc7a9e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.387216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.389246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 216/449] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/groups/f0adfa3c32634b93b51fb7738dc7a9e8 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:21.395053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.401261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9eb587-be68-4eed-bf77-56ca62131032 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.440247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.440936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.441272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.441554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.442338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:21.442989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.444224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.454574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 234/450] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:21.460864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.516954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f4c7c-eab5-4387-b2cb-969de1fe4ef5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.540158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:21.552841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.553764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.554049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.554326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.555150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] RBAC: Adding query filter params (group.id=f0adfa3c32634b93b51fb7738dc7a9e8, role.id=8719b7bceaea49a48ff010ff8d107832, scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:21.555462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.556900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.562562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 217/451] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/role_assignments?group.id=f0adfa3c32634b93b51fb7738dc7a9e8&scope.project.id=dbc8a47ac1834033aef7a1d6151bd94e&role.id=8719b7bceaea49a48ff010ff8d107832 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:21.563683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88fd26-6c18-4ea0-8ab0-77f9df05f8b0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.591490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:21.602925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.603592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.603875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.604152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.677403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.678881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.785528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 235/452] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:42:21.791142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.894773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f290cdf-18cd-4437-ad5c-46dfd64c69c9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:21.932292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:21.932508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:21.932795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:21.933074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] PATH_INFO: `/v3/projects/e741295126a84576b8368706b7ffe76d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:21.934304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=e741295126a84576b8368706b7ffe76d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:21.935748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:21.937502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 218/453] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/projects/e741295126a84576b8368706b7ffe76d => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:21.943610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:21.968961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:22.049454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dae31c-612b-42d6-b392-b237b2bf275b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:22.083970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7bc5d-5ecc-4bff-8a33-20f60da3b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:22.084842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7bc5d-5ecc-4bff-8a33-20f60da3b141 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:22.085121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7bc5d-5ecc-4bff-8a33-20f60da3b141 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:22.085422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7bc5d-5ecc-4bff-8a33-20f60da3b141 None admin] PATH_INFO: `/v3/groups/bec73769607c49529a8df1aaa170f62a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:22.086602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7bc5d-5ecc-4bff-8a33-20f60da3b141 None admin] RBAC: Authorizing `identity:get_group(group_id=bec73769607c49529a8df1aaa170f62a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:22.088038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7bc5d-5ecc-4bff-8a33-20f60da3b141 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:22.089722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 236/454] 10.222.0.91 () {56 vars in 1180 bytes} [Thu May 26 13:42:21 2022] GET /identity/v3/groups/bec73769607c49529a8df1aaa170f62a => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:22.095459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:22.148898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:22.195280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e992817-0413-4dac-a0b7-9080cd829cb7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:22.228973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4be41-dd95-401d-b469-7fdc4c13b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:22.229659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4be41-dd95-401d-b469-7fdc4c13b951 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:22.229945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4be41-dd95-401d-b469-7fdc4c13b951 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:22.230220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4be41-dd95-401d-b469-7fdc4c13b951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:22.231055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4be41-dd95-401d-b469-7fdc4c13b951 None admin] RBAC: Adding query filter params (group.id=bec73769607c49529a8df1aaa170f62a, role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=e741295126a84576b8368706b7ffe76d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:22.231370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4be41-dd95-401d-b469-7fdc4c13b951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:22.232802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4be41-dd95-401d-b469-7fdc4c13b951 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:22.244287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 219/455] 10.222.0.91 () {56 vars in 1376 bytes} [Thu May 26 13:42:22 2022] GET /identity/v3/role_assignments?group.id=bec73769607c49529a8df1aaa170f62a&scope.project.id=e741295126a84576b8368706b7ffe76d&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:24.878467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fae90-2d5a-4a12-a9b4-133224ba94ff None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:24.878996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fae90-2d5a-4a12-a9b4-133224ba94ff None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:24.878996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fae90-2d5a-4a12-a9b4-133224ba94ff None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:24.880416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 237/456] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:24.889572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8ddfe-a04a-4d3e-97c1-ab7f10d6b0c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:24.890092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8ddfe-a04a-4d3e-97c1-ab7f10d6b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:24.890167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8ddfe-a04a-4d3e-97c1-ab7f10d6b0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:24.918315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e0b8ddfe-a04a-4d3e-97c1-ab7f10d6b0c3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:24.919626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e0b8ddfe-a04a-4d3e-97c1-ab7f10d6b0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:25.005529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 220/457] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:25.047791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fae90-2d5a-4a12-a9b4-133224ba94ff None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:25.077943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53f440-4b3f-4c53-9748-ecc78205fe43 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.078274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53f440-4b3f-4c53-9748-ecc78205fe43 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.078491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53f440-4b3f-4c53-9748-ecc78205fe43 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.079941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 221/458] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:42:25.085723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a474-ccbc-48fe-9040-eb0f57cde7f1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.086019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a474-ccbc-48fe-9040-eb0f57cde7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.086302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a474-ccbc-48fe-9040-eb0f57cde7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.122203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-90d1a474-ccbc-48fe-9040-eb0f57cde7f1 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:25.123464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90d1a474-ccbc-48fe-9040-eb0f57cde7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:25.172820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:25.196050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fae90-2d5a-4a12-a9b4-133224ba94ff None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:25.213471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:25.266154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 222/459] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:25.279024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:25.279921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.280205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.280484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.289034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1a474-ccbc-48fe-9040-eb0f57cde7f1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:25.296688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:25.299570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:25.329730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:25.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:25.331523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:25.331523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:25.331523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:25.331523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:25.337624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 238/460] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:25.343305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:25.405058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1a474-ccbc-48fe-9040-eb0f57cde7f1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:25.442554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf236647-898b-4bbf-bfe1-c37bf0c36941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:25.443901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf236647-898b-4bbf-bfe1-c37bf0c36941 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.444186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf236647-898b-4bbf-bfe1-c37bf0c36941 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.444464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf236647-898b-4bbf-bfe1-c37bf0c36941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.456310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf236647-898b-4bbf-bfe1-c37bf0c36941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:25.457818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf236647-898b-4bbf-bfe1-c37bf0c36941 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bf236647-898b-4bbf-bfe1-c37bf0c36941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:25.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:25.477653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:25.478499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:25.478499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:25.478499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:25.478499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:25.478499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 223/461] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:25.479597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53667-600c-4086-9176-efe2a4b86f0f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.479858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53667-600c-4086-9176-efe2a4b86f0f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.480136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53667-600c-4086-9176-efe2a4b86f0f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.486375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 224/462] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:25.490533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53667-600c-4086-9176-efe2a4b86f0f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:25.511921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad38b2-1745-4960-8480-04efa51df180 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:25.538822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:25.550802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73256f9-c210-4fe5-ba21-2b8b5fceec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:25.552136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73256f9-c210-4fe5-ba21-2b8b5fceec9a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.552633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73256f9-c210-4fe5-ba21-2b8b5fceec9a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.553330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73256f9-c210-4fe5-ba21-2b8b5fceec9a None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.554190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73256f9-c210-4fe5-ba21-2b8b5fceec9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:25.554666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73256f9-c210-4fe5-ba21-2b8b5fceec9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:25.556065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73256f9-c210-4fe5-ba21-2b8b5fceec9a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:25.568307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 239/463] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:25.573831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862600a-abfe-4cca-90aa-773c440d476a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.574108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862600a-abfe-4cca-90aa-773c440d476a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.574389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862600a-abfe-4cca-90aa-773c440d476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.580270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:25.592326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2862600a-abfe-4cca-90aa-773c440d476a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:25.593592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2862600a-abfe-4cca-90aa-773c440d476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:25.623473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53667-600c-4086-9176-efe2a4b86f0f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:25.668711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 240/464] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:25.670137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9d0bc-a526-4135-b0dd-40580116e01f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.670429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9d0bc-a526-4135-b0dd-40580116e01f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.670704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9d0bc-a526-4135-b0dd-40580116e01f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.672092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 241/465] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:25.672677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9d0bc-a526-4135-b0dd-40580116e01f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:25.682356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82894e98-24fe-4171-88df-749116a521ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:25.683253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894e98-24fe-4171-88df-749116a521ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.683550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894e98-24fe-4171-88df-749116a521ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.683883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894e98-24fe-4171-88df-749116a521ff None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.684617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894e98-24fe-4171-88df-749116a521ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:25.684825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894e98-24fe-4171-88df-749116a521ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:25.688148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894e98-24fe-4171-88df-749116a521ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:25.699924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 225/466] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:25.701251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922b92f-ba4d-4bef-8543-766172d8a89e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.701546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922b92f-ba4d-4bef-8543-766172d8a89e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.701822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922b92f-ba4d-4bef-8543-766172d8a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.718189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f922b92f-ba4d-4bef-8543-766172d8a89e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:25.719985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f922b92f-ba4d-4bef-8543-766172d8a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:25.787087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9d0bc-a526-4135-b0dd-40580116e01f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:25.789997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 226/467] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:25.790827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922b92f-ba4d-4bef-8543-766172d8a89e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:25.815453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:25.841070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:25.842274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.843154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.843666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.844823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:25.846965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:25.856150 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:25.885183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 242/468] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:42:25 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:25.885183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:25.913188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922b92f-ba4d-4bef-8543-766172d8a89e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:25.977092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:25.978586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:25.978995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:25.979278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:25.996030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:25.998101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:26.020107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:26.021120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:26.022036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:26.022036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:26.022036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:26.022036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:26.026635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 227/469] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:26.027759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:26.034911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e81175-5e6c-42e7-ae5e-771cf2c35871 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:26.084179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:26.084861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.085140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.085466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.086174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:26.087606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.111343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 243/470] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:42:25 2022] POST /identity/v3/users => generated 248 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:26.112567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:26.127310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:26.171539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e33e8-5416-495f-9a45-fe71acdba758 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:26.182175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:26.233669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:26.234262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.234545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.234819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.250396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:26.252234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.262213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf78cb1-1140-4cc0-8423-51888b3f3945 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:26.277510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:26.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:26.279547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:26.279547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:26.279547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:26.279547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:26.279547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 228/471] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:26.284973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:26.312439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:26.313775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.314277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.314735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.315916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:26.316455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:26.318536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.338296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 244/472] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:26.344087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:26.442044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cb4b3-3dc2-43a0-b6ea-293b781e56d3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:26.471886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:26.476753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0ecb9-966f-4288-a1cb-18c016e4ba3c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:26.485497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:26.486310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:26.486989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.487315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.487725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.489329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:26.489667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:26.491198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.504551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 229/473] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:26.511760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:26.515095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520e23d-e410-401f-82b6-b289c87235af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:26.515836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520e23d-e410-401f-82b6-b289c87235af None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.516172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520e23d-e410-401f-82b6-b289c87235af None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.516566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520e23d-e410-401f-82b6-b289c87235af None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.518035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520e23d-e410-401f-82b6-b289c87235af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:26.518768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520e23d-e410-401f-82b6-b289c87235af None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.545030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 245/474] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:42:26 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:42:26.639118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc75d91-7c8d-4ae8-91af-cdefbf9c7249 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.639413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc75d91-7c8d-4ae8-91af-cdefbf9c7249 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.639696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc75d91-7c8d-4ae8-91af-cdefbf9c7249 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.641472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 246/475] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:42:26.649112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dd1c9-cf2e-4135-b1ae-337850cf92ce None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.649432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dd1c9-cf2e-4135-b1ae-337850cf92ce None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.649714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dd1c9-cf2e-4135-b1ae-337850cf92ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.651154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f0279-13fa-4635-a29c-47a0fd7129c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:26.668602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ab0dd1c9-cf2e-4135-b1ae-337850cf92ce None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:26.669954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab0dd1c9-cf2e-4135-b1ae-337850cf92ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:26.699701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:26.701411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.701871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.702556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.703723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:26.706250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.735687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 230/476] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:42:26 2022] POST /identity/v3/users => generated 249 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:42:26.768386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 247/477] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:26.793312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:26.859334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:26.940799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878f788-3e11-451d-9308-f8c1804b8dae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:26.984399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:26.985170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:26.985474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:26.985835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:26.986518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:26.987915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:26.996167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 13:42:26.996167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 13:42:26.996167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [parameters: {'id': '4be83f9826ce474fb200fbb91a8462df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 13:42:26.996167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 26 13:42:27.013859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return ref.to_dict() May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:42:27.016043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 13:42:27.018124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 13:42:27.019756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:42:27.024075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:42:27.026550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': '4be83f9826ce474fb200fbb91a8462df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:27.028437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 13:42:27.030355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:27.032338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 231/478] 10.222.0.91 () {60 vars in 1131 bytes} [Thu May 26 13:42:26 2022] POST /identity/v3/roles => generated 144 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:27.032469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dd1c9-cf2e-4135-b1ae-337850cf92ce None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:27.137594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:27.148270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dd1c9-cf2e-4135-b1ae-337850cf92ce None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:27.191120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226462f-bad0-40b1-a2a5-3a0bb2ea0dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:27.191959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226462f-bad0-40b1-a2a5-3a0bb2ea0dde None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:27.192242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226462f-bad0-40b1-a2a5-3a0bb2ea0dde None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:27.192545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226462f-bad0-40b1-a2a5-3a0bb2ea0dde None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:27.211814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226462f-bad0-40b1-a2a5-3a0bb2ea0dde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e226462f-bad0-40b1-a2a5-3a0bb2ea0dde None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:27.220146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:27.222338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:27.224350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 248/479] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:42:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:27.227572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:27.366439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e4c2e-0bd9-46a3-80ce-9f0043701e58 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:27.369255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:27.412662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedab41-1775-40e2-97b9-91f07a0e9c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:27.413736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedab41-1775-40e2-97b9-91f07a0e9c57 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:27.414219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedab41-1775-40e2-97b9-91f07a0e9c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:27.414708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedab41-1775-40e2-97b9-91f07a0e9c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:27.415653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedab41-1775-40e2-97b9-91f07a0e9c57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:27.416162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedab41-1775-40e2-97b9-91f07a0e9c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:27.417787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedab41-1775-40e2-97b9-91f07a0e9c57 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:27.428276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 232/480] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:28.798411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ad78-cb70-4e1b-8ab3-bd0b5832cd40 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:28.798934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ad78-cb70-4e1b-8ab3-bd0b5832cd40 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:28.799009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ad78-cb70-4e1b-8ab3-bd0b5832cd40 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:28.800575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 249/481] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:28.806604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e735df-fdf8-4053-bfa2-6f822af4f03a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:28.807019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e735df-fdf8-4053-bfa2-6f822af4f03a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:28.807186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e735df-fdf8-4053-bfa2-6f822af4f03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:28.829349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a5e735df-fdf8-4053-bfa2-6f822af4f03a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:28.830545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5e735df-fdf8-4053-bfa2-6f822af4f03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:28.919114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 233/482] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:28.951070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ad78-cb70-4e1b-8ab3-bd0b5832cd40 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:29.097955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ad78-cb70-4e1b-8ab3-bd0b5832cd40 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:29.147126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:29.147943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.148226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.148505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.157794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b3771-f9c1-4a0e-bfb6-1a1db9a13899 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.158815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b3771-f9c1-4a0e-bfb6-1a1db9a13899 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.159866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b3771-f9c1-4a0e-bfb6-1a1db9a13899 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.167812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 234/483] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:29.168403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:29.170106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038ee6a-06e8-4465-97c5-19f62c4a79b7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.170598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038ee6a-06e8-4465-97c5-19f62c4a79b7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.171091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038ee6a-06e8-4465-97c5-19f62c4a79b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:29.181860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:29.182807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:29.184218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 250/484] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:29.188804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:29.213025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4038ee6a-06e8-4465-97c5-19f62c4a79b7 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:29.216553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4038ee6a-06e8-4465-97c5-19f62c4a79b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:29.237531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:29.350283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 235/485] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:29.370738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8ffc-22a1-48a9-81d0-c333767bcc24 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:29.382068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038ee6a-06e8-4465-97c5-19f62c4a79b7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:29.424413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:29.425662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.427304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.428532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.429570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:29.430702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:29.432468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:29.434984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:29.452943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 251/486] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:29.461416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:29.555338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038ee6a-06e8-4465-97c5-19f62c4a79b7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:29.597707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cbff7-59b1-4fd0-bf5b-f261156f550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:29.598520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cbff7-59b1-4fd0-bf5b-f261156f550e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.599609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cbff7-59b1-4fd0-bf5b-f261156f550e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.599888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cbff7-59b1-4fd0-bf5b-f261156f550e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.611665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:29.620041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cbff7-59b1-4fd0-bf5b-f261156f550e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-726cbff7-59b1-4fd0-bf5b-f261156f550e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:29.632008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:29.633817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 236/487] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:29.635993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e701e3b-238a-4ce0-a328-f1c674876033 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.637959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e701e3b-238a-4ce0-a328-f1c674876033 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.637959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e701e3b-238a-4ce0-a328-f1c674876033 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.637959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 237/488] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:29.647914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e701e3b-238a-4ce0-a328-f1c674876033 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:29.655302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21e6aa-153d-4962-ba93-b669ea688873 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:29.701810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b8ca7-6798-442e-a403-7493c5c24493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:29.702649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8ca7-6798-442e-a403-7493c5c24493 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.702933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8ca7-6798-442e-a403-7493c5c24493 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.703205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8ca7-6798-442e-a403-7493c5c24493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.718465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b8ca7-6798-442e-a403-7493c5c24493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:29.720084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b8ca7-6798-442e-a403-7493c5c24493 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7a6b8ca7-6798-442e-a403-7493c5c24493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:29.739376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:29.741219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:29.743229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:29.743229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:29.743229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:29.743229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:29.743229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 252/489] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:29.743857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a487f-e6ec-4480-9191-72d37dc478e7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.743857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a487f-e6ec-4480-9191-72d37dc478e7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.744127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a487f-e6ec-4480-9191-72d37dc478e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.758987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:29.768607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a63a487f-e6ec-4480-9191-72d37dc478e7 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:29.769941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a63a487f-e6ec-4480-9191-72d37dc478e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:29.779162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e701e3b-238a-4ce0-a328-f1c674876033 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:29.826453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:29.827249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.827449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.827728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.828535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:29.828890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:29.830358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:29.843139 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 238/490] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:29.845205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:29.871047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 253/491] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:29.872036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60206-b6ce-400f-b14b-a4776d0c750d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:29.872400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60206-b6ce-400f-b14b-a4776d0c750d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:29.872756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60206-b6ce-400f-b14b-a4776d0c750d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:29.875928 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 254/492] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:29.881218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60206-b6ce-400f-b14b-a4776d0c750d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:29.952030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:29.990723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179467a7-7b4a-4fc7-83c9-326232909baf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.002251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60206-b6ce-400f-b14b-a4776d0c750d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.032794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f805d3-3c79-4b50-ba33-33c12f8378d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.033494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f805d3-3c79-4b50-ba33-33c12f8378d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.033872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f805d3-3c79-4b50-ba33-33c12f8378d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.034062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f805d3-3c79-4b50-ba33-33c12f8378d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.034785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f805d3-3c79-4b50-ba33-33c12f8378d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:30.035093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f805d3-3c79-4b50-ba33-33c12f8378d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.036488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f805d3-3c79-4b50-ba33-33c12f8378d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:30.048920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.049742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 239/493] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:30.049875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.050069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.050315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.054715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a3f36-d01a-43e2-967b-2ac7abb27893 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.055007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a3f36-d01a-43e2-967b-2ac7abb27893 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.055291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a3f36-d01a-43e2-967b-2ac7abb27893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.063128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.064891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:30.072660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7e5a3f36-d01a-43e2-967b-2ac7abb27893 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:30.073705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e5a3f36-d01a-43e2-967b-2ac7abb27893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:30.076629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:30.087235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:30.088887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:30.090156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:30.090156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:30.090156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:30.090156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:30.090156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 255/494] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:30.090156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.159842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 240/495] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:30.161669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a3f36-d01a-43e2-967b-2ac7abb27893 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.240585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7e992-ca45-471d-bb3f-d004d422fec6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.250577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:30.283594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.284394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.284674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.284945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.296296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:30.299436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a3f36-d01a-43e2-967b-2ac7abb27893 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:30.305129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:30.306336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 256/496] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:30.307164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.348802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.349644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.349916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.350188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.360199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.361684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:30.363446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:30.380889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:30.382030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 241/497] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:30.382974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.423842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba51bcf-eada-4717-9324-63ddc3093c3b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.467439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.468118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.468397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.468671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.469817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:30.469817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.471103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:30.484799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 257/498] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:30.485657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.506839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68861d8d-299e-42ed-8f97-881109f78157 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.515736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:30.542374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8c3a0-285f-4ad9-9de3-2f0607e42a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.543168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8c3a0-285f-4ad9-9de3-2f0607e42a40 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.543452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8c3a0-285f-4ad9-9de3-2f0607e42a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.543730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8c3a0-285f-4ad9-9de3-2f0607e42a40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.554949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8c3a0-285f-4ad9-9de3-2f0607e42a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cbd8c3a0-285f-4ad9-9de3-2f0607e42a40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:30.563360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:30.565258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 242/499] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:30.566737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949e53-9018-4b77-9f1e-fad0a71ad3d4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.570482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949e53-9018-4b77-9f1e-fad0a71ad3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.570482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0949e53-9018-4b77-9f1e-fad0a71ad3d4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.570482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 243/500] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:30.570482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0949e53-9018-4b77-9f1e-fad0a71ad3d4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.597925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79878c-eaed-463a-8351-6f6dcbcef222 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.631066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:30.633688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.634541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.634822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.635092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.635839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:30.636144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.637561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:30.645687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 258/501] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:30.646486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.677815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0949e53-9018-4b77-9f1e-fad0a71ad3d4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.715749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a4f9b-35c3-4888-ac21-1fa2357bf880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.716429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4f9b-35c3-4888-ac21-1fa2357bf880 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.716709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4f9b-35c3-4888-ac21-1fa2357bf880 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.716985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4f9b-35c3-4888-ac21-1fa2357bf880 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.717855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a4f9b-35c3-4888-ac21-1fa2357bf880 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:30.718177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a4f9b-35c3-4888-ac21-1fa2357bf880 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.719648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a4f9b-35c3-4888-ac21-1fa2357bf880 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:30.747303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:30.760906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d3a1-9ddb-4331-8063-27280d67613e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.765935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 244/502] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:30.768352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e9ef2-59f3-4107-a80c-fe3111841571 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.768743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e9ef2-59f3-4107-a80c-fe3111841571 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.768912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e9ef2-59f3-4107-a80c-fe3111841571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.787052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-678e9ef2-59f3-4107-a80c-fe3111841571 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:30.788241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-678e9ef2-59f3-4107-a80c-fe3111841571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:30.809143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.810139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.810418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.810692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.819159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.820603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:30.836409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:30.837390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:30.838346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 259/503] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:30.839059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.865841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 245/504] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:30.867313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e9ef2-59f3-4107-a80c-fe3111841571 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:30.929439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:30.956484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b820011-eb14-4625-b3a1-4289d437d786 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.987883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e9ef2-59f3-4107-a80c-fe3111841571 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:30.992530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:30.993228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:30.993515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:30.993790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:30.994531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:30.994836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:30.996228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.007693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 260/505] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:31.008579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.025104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:31.031559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.032362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.032645 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.032920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.044879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.046385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:31.065283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:31.066402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:31.067301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:31.067301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:31.067301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.067301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:31.067301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 246/506] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:31.067301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.125454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7fed7-cbae-41e1-97ff-79f66dcce2d2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.170512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.171156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.171437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.171713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.178464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:31.185044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e885b0-ba97-4737-9a93-c0693cbb2252 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.224500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.225123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.225469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.225750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.226698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:31.227222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.228835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.249462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 247/507] 10.222.0.91 () {56 vars in 1137 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:31.250329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.258454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.259954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.274872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:31.368478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d6e8e-98b8-40d2-9172-47c86cb850b4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.383738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 261/508] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:31.384910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.410772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.412440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.412743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.412990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.426089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.428279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:31.449321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:31.450392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:31.451502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:31.451502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:31.451502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.451502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:31.451502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 248/509] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:31.452541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.471027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:31.512645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e108a-35d4-4fc1-91b4-d05ef21450cd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.527782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:31.552584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.553413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.553699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.553981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.565508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.567014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.581009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba448c-96d1-4c3c-b3e7-2c0d4de864b5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:31.585871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:31.586946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:31.587856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:31.587856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:31.587856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.587856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:31.587856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 262/510] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:31.588282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.623106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.624060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.624512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.624879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.625769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:31.626171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.627938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.644769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 249/511] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:31.646799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.699762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b9aaf-9730-43e8-9345-0132f53d2ff7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.719551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:31.745675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.746709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.747183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.747626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.771013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49495f2-7b83-48b7-9e66-9b00d3fd4daa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.790388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:31.811351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:31.812162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:31.812462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:31.812741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:31.824539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.826063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.840799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:31.843400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:31.844090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:31.845111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:31.852574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:31.852574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:31.852574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:31.852574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:31.852574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 250/512] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:31.852574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.957682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 263/513] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:31.959587 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:31.977077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3f1ca-8370-4e3b-a7d9-ee7f2d2dc092 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:31.997035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.013666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.014332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.014618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.014893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.015624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:32.015929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.017601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.027853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.030666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 251/514] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:32.031554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.071983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f37ff5-a891-450c-b7ec-46a04736812d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.112008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.112667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.112954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.113247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.113966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:32.114298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.115735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.127431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 264/515] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:32.128075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.142375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2c871-e837-404a-8672-16bcf751f3f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.180826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.181668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.181955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.182264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.194228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.196113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:32.212788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:32.213920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:32.214988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 252/516] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:32.216541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.249585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bb4a4-9b3e-4aba-9942-b928aee4ffab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.257572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.273234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.299343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.300017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.300301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.300579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.301339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:32.301645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.303057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.316229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 265/517] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:32.317031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.319455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25b3c9-3ba7-4be2-a9a1-3c871f37d2f8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.361686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.362481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.362763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.363036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.374011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.375499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:32.394084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:32.394983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:32.395897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:32.395897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:32.395897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:32.395897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:32.395897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 253/518] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:32.396166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.440557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08b079-3aaa-4d9e-ba6d-20e35239ed4e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.481967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.483171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.483457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.484005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.484809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.487335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.515177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 266/519] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:42:32 2022] POST /identity/v3/users => generated 247 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:32.516006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.532294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.544450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c571d-44af-422d-bba1-0d13b53a726c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.572164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.599245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261107-193f-450a-a003-fd149862c94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.600370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261107-193f-450a-a003-fd149862c94a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.600676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261107-193f-450a-a003-fd149862c94a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.600917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261107-193f-450a-a003-fd149862c94a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.615799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261107-193f-450a-a003-fd149862c94a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.617520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261107-193f-450a-a003-fd149862c94a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-57261107-193f-450a-a003-fd149862c94a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:32.637616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:32.639023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:42:32.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:32.644851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 254/520] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:32.645800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261107-193f-450a-a003-fd149862c94a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.666761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca1135-4b59-4de2-b706-6e1693a68a45 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.725465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.726156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.726435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.726811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.728266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:32.728266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.729384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.770963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 267/521] 10.222.0.91 () {56 vars in 1137 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:32.772134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.792754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261107-193f-450a-a003-fd149862c94a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.834545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.834918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.834918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.835181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.843407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.844913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:32.864400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:32.865946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 255/522] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:32.872034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:32.891339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.903439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:32.909866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d0c31-8f53-45e5-b3f6-94e4da9b6213 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:32.955045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:32.955716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:32.956028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:32.956250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:32.957070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:32.957419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:32.959367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:32.972383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 268/523] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:32.973260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:33.032625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720aa54-cd53-4aea-9cd7-94e0fccffe41 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:33.073834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:33.074578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:33.074944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:33.075218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:33.076257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:33.076382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:33.079551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:33.110777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 256/524] 10.222.0.91 () {56 vars in 1139 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:33.111828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:33.117423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e37ba-d9b1-49aa-ac01-3b3616c2d193 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:33.159778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:33.160427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:33.160707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:33.160977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:33.196763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:33.197652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:33.245028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3503a-9c71-4a36-b150-f222a0dfba70 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:33.254084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:33.256127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:33.298547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:33.299464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:33.299817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:33.300219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:33.301028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:33.301460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:33.303414 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:33.332864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 257/525] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:33.333725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:33.408877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 269/526] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:33.409781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:33.467078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25672f8-a5a3-4a7a-8b5f-1edbde27559d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:33.507120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:33.512119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:33.521019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:33.522189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:33.522623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:33.523219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:33.533721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:33.535479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:33.550998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:33.552928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:33.554186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 258/527] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:33.555057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:33.573845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08c1c0-c0ff-4d42-8eff-2225625a6bce None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:33.618847 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:33.619441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:33.619695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:33.619964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:33.712292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65ce6-428d-4bf7-9ea4-00065b37f19f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:33.727668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:33.729287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:33.761699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:33.762440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:33.762807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:33.763170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:33.763832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=f579f2db041f455786a02dcf621619ed) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:33.764330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:33.765468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:33.771769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 259/528] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:33 2022] GET /identity/v3/role_assignments?user.id=f579f2db041f455786a02dcf621619ed&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:33.772538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:33.826303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:33.852213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:33.883591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 270/529] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:33.884859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:33.937036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b6a3d-193a-41f0-b5f5-e79dcb929216 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.001114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70121c-e3b0-44ad-8d62-2530a5650265 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.002335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.003897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.004364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.004840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.027159 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.029910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.042471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.043513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.044000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.044478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.045578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:34.046093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.048006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:34.057214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:34.058445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:34.059323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:34.059323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:34.059323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:34.059323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:34.059323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 260/530] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:34.059608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.078550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 271/531] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:34.079965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.104064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:34.203083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8fb5-923f-416a-baba-6bdd397606d4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.213119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:34.228617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88544c-0020-4cbf-919b-92e389e02ac4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.254228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.255125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.255438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.256008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.269933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.271807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.276412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.277102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.277417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.277695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.278436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:34.278741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.280264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:34.293832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:34.294805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:34.296163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:34.296163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:34.296163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:34.296163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:34.296419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 272/532] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:34.296691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.298712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 261/533] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:34.302615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.420394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:34.444168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6170d-9cd9-4481-9715-6be51c476956 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.446557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00529477-39d3-4ea2-b001-27432d12f07c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.488249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.489892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.490359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.490774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.491827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=a0c209755dfc4b4089176a056cf2f683) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:34.492321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.493961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.498110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.498915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.499277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.499668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.501970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 273/534] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/role_assignments?user.id=a0c209755dfc4b4089176a056cf2f683&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:34.503025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.524308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.534915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:34.560428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:34.561485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:34.562642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:34.574688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 262/535] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:34.582304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.617481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:34.647181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb20d49-347f-4c9b-a70c-31aca56252f4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.688324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.689340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.689779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.690232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.691142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:34.691599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.693271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.707586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 274/536] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:34.714139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.715616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7529d-ac4e-4b53-9ead-d26168372057 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.717565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:34.757630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.757861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.758069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.758342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.759145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:34.759457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.760917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.775154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 263/537] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:34.786346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.835118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450c7f7-3d2e-47f9-b1f0-85934b44fa1d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.875278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.876667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.877193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.877663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.896632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.899252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.917595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:34.939179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:34.940068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:34.940918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:34.944795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb83b4-9a1a-4b89-be05-9ba735a14dee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:34.946299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 275/538] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:34.952139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:34.985186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f58f5-9c62-4b46-a552-0e541bb32353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:34.985186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f58f5-9c62-4b46-a552-0e541bb32353 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.985186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f58f5-9c62-4b46-a552-0e541bb32353 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.985530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f58f5-9c62-4b46-a552-0e541bb32353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.986500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f58f5-9c62-4b46-a552-0e541bb32353 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=8a381dccf1ca4dacaab4e0ae0d473525) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:34.986815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f58f5-9c62-4b46-a552-0e541bb32353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:34.988761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f58f5-9c62-4b46-a552-0e541bb32353 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:34.997311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 264/539] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/role_assignments?user.id=8a381dccf1ca4dacaab4e0ae0d473525&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:34.998649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899775-9d25-45c2-84d0-42071b55a14e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.998944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899775-9d25-45c2-84d0-42071b55a14e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.999220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8899775-9d25-45c2-84d0-42071b55a14e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.001439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 265/540] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:35.010371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0caaca-1801-4938-b1ba-abb6508ecc3f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:35.011381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0caaca-1801-4938-b1ba-abb6508ecc3f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:35.012196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0caaca-1801-4938-b1ba-abb6508ecc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.034744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6c0caaca-1801-4938-b1ba-abb6508ecc3f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:35.035111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:35.038070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c0caaca-1801-4938-b1ba-abb6508ecc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:35.103186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72574f63-8e43-45aa-8556-049bf7caa02b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:35.173134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 266/541] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:35.187481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:35.188428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:35.188858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:35.189399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.190364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:35.190829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:35.192538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:35.212903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 276/542] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:35.218983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0caaca-1801-4938-b1ba-abb6508ecc3f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:35.248674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:35.356356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:35.393804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058edb2f-1626-47f1-a970-cebe88e76726 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:35.421796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:35.439326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:35.441804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:35.443223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:35.443709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.449014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0caaca-1801-4938-b1ba-abb6508ecc3f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:35.470658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:35.480546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:35.481489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:35.482311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:35.485432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 277/543] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:35.492823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:35.531715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:35.532389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:35.532676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:35.532958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.533842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=79d624b7de50480bb4002eb23ec2b360) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:35.534161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:35.540123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:35.553905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 267/544] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:35 2022] GET /identity/v3/role_assignments?user.id=79d624b7de50480bb4002eb23ec2b360&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:35.689403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f2e8f-3486-40db-b4ef-c4ee8b02297f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:35.744838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:35.745474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:35.745760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:35.746039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.746807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:35.747116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:35.748541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:35.761809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 278/545] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:35.767510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:35.914536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a6678-9b9d-46b0-9f5a-4af67eb2adb5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:35.963462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:35.964268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:35.964557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:35.964836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.970211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:35.985421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:35.986924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:36.010755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:36.011683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:36.012579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:36.012579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:36.012579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:36.012579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:36.012579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 268/546] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:36.017624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:36.105108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:36.204628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c35de-83d3-4068-9c80-e13641bb23ce None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:36.251924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:36.252599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.252889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.253188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.253929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:36.254240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:36.256342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:36.269361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 279/547] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:36.274878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:36.426811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e307-4595-477b-b6b0-19bab2ade424 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:36.466984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d83c0-4686-45fc-ac28-723aababb6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:36.467789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d83c0-4686-45fc-ac28-723aababb6ec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.468068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d83c0-4686-45fc-ac28-723aababb6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.468341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d83c0-4686-45fc-ac28-723aababb6ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.476698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d83c0-4686-45fc-ac28-723aababb6ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:36.478191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d83c0-4686-45fc-ac28-723aababb6ec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ea4d83c0-4686-45fc-ac28-723aababb6ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:36.497140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:36.498649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:36.499749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 269/548] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:36.505830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:36.523218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a28da-6199-4243-a11a-d112c3b35fda None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.523411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a28da-6199-4243-a11a-d112c3b35fda None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.523699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a28da-6199-4243-a11a-d112c3b35fda None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.526465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 270/549] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:42:36.530694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea11c5-03bc-4639-a174-24033a1f8f3b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.530986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea11c5-03bc-4639-a174-24033a1f8f3b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.531270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea11c5-03bc-4639-a174-24033a1f8f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.561393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cdea11c5-03bc-4639-a174-24033a1f8f3b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:36.562592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cdea11c5-03bc-4639-a174-24033a1f8f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:36.657867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:36.689267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 271/550] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:36.709700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea11c5-03bc-4639-a174-24033a1f8f3b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:36.716984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc45455-3f3b-4398-a612-5da0d3ccb4a8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:36.722128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:36.805705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805140-4122-488c-81e4-9609073db421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:36.806315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805140-4122-488c-81e4-9609073db421 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.806603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805140-4122-488c-81e4-9609073db421 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.806943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805140-4122-488c-81e4-9609073db421 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.807801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f805140-4122-488c-81e4-9609073db421 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:36.808113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f805140-4122-488c-81e4-9609073db421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:36.809593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f805140-4122-488c-81e4-9609073db421 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:36.818110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea11c5-03bc-4639-a174-24033a1f8f3b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:36.848879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 280/551] 10.222.0.91 () {56 vars in 1135 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:36.854064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805140-4122-488c-81e4-9609073db421 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:36.855244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:36.856104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.856553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.856839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.869508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:36.877648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:36.878659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:36.879565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:36.884254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 272/552] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:36.885495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:36.932358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:37.000022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d91ef-7271-4841-a87f-cb17d11127dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.036358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.037061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.037383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.037669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.038433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:37.038744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:37.040144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:37.054344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 273/553] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:37.055635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:37.135153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805140-4122-488c-81e4-9609073db421 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.167841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:37.180229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19de57-0b83-451d-bac6-a4e6506527fc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.195140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9b096-b96e-4441-980f-ba890661b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.196173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b096-b96e-4441-980f-ba890661b44d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.196173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b096-b96e-4441-980f-ba890661b44d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.196375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b096-b96e-4441-980f-ba890661b44d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.251366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.252188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.252474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.252749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.273531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:37.278612 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:37.280089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:37.311671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:37.313746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:37.315767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:37.315767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:37.315767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:37.315767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:37.315767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 274/554] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:37.322295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:37.342974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9b096-b96e-4441-980f-ba890661b44d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:37.344465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9b096-b96e-4441-980f-ba890661b44d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:37.469920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f0546-66ba-43c8-8f18-95505347811a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.486398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 281/555] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:42:37.488091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b878ad-5e5a-4b44-9747-b3bbde8c6b52 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.488541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b878ad-5e5a-4b44-9747-b3bbde8c6b52 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.488975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b878ad-5e5a-4b44-9747-b3bbde8c6b52 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.491229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 282/556] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:37.492738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b878ad-5e5a-4b44-9747-b3bbde8c6b52 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:37.510989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e13d54-e3cb-4b99-aa7f-c2e9e261c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.511973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13d54-e3cb-4b99-aa7f-c2e9e261c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.512459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13d54-e3cb-4b99-aa7f-c2e9e261c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.512931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13d54-e3cb-4b99-aa7f-c2e9e261c8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.513937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e13d54-e3cb-4b99-aa7f-c2e9e261c8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:37.514474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e13d54-e3cb-4b99-aa7f-c2e9e261c8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:37.516113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e13d54-e3cb-4b99-aa7f-c2e9e261c8e5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:37.530408 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 275/557] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:37.532062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb93a-056a-4d77-b6e5-48e8c0ee418b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.532547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb93a-056a-4d77-b6e5-48e8c0ee418b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.533094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb93a-056a-4d77-b6e5-48e8c0ee418b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.552233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c5dfb93a-056a-4d77-b6e5-48e8c0ee418b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:37.553840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c5dfb93a-056a-4d77-b6e5-48e8c0ee418b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:37.562258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:37.569879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:37.616542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b878ad-5e5a-4b44-9747-b3bbde8c6b52 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.626531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 276/558] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:37.627681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfb93a-056a-4d77-b6e5-48e8c0ee418b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:37.654867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.656044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.656587 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.657037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.669835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:37.671611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:37.692062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:37.693580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:37.694519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:37.694519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:37.694519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:37.694519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:37.696793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 283/559] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:37.697919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:37.739670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfb93a-056a-4d77-b6e5-48e8c0ee418b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.778962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.780581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.780869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.781169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.790963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:37.793208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:37.811910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859eb97-c464-45ae-983e-50377c83c0d8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:37.813586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:37.814573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:37.815623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:37.816366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 277/560] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:42:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:37.817324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:37.820279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:37.843192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:37.866006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117b765-873f-45ec-8cd8-50224dfbce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.867532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117b765-873f-45ec-8cd8-50224dfbce38 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.868149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117b765-873f-45ec-8cd8-50224dfbce38 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.868435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117b765-873f-45ec-8cd8-50224dfbce38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.882998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117b765-873f-45ec-8cd8-50224dfbce38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7117b765-873f-45ec-8cd8-50224dfbce38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:37.893418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:37.894370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:37.895215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:37.898187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 284/561] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:37.898187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074fd0-8b9f-4675-b162-9a486674163f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.898187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074fd0-8b9f-4675-b162-9a486674163f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.899908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074fd0-8b9f-4675-b162-9a486674163f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.902031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 285/562] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:37.902259 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37074fd0-8b9f-4675-b162-9a486674163f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:37.941823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ea693-affc-4ce0-a643-1a833fe833a8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:37.985020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:37.985734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:37.986014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:37.986519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:37.987255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:37.987565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:37.988935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.007224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 278/563] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:38.008043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.015111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37074fd0-8b9f-4675-b162-9a486674163f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.051763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883eced-5a30-4a43-8d89-1ca7d2103429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.052453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883eced-5a30-4a43-8d89-1ca7d2103429 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.052738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883eced-5a30-4a43-8d89-1ca7d2103429 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.053040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883eced-5a30-4a43-8d89-1ca7d2103429 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.053900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883eced-5a30-4a43-8d89-1ca7d2103429 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:38.054211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883eced-5a30-4a43-8d89-1ca7d2103429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.055665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883eced-5a30-4a43-8d89-1ca7d2103429 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.075630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.084313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 286/564] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:42:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:38.084313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28260-eb3f-46c9-8a60-bda1e23a8852 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.084313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28260-eb3f-46c9-8a60-bda1e23a8852 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.084313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28260-eb3f-46c9-8a60-bda1e23a8852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.102742 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c9e28260-eb3f-46c9-8a60-bda1e23a8852 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:38.103921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c9e28260-eb3f-46c9-8a60-bda1e23a8852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:38.126465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff8643-6f03-4300-997f-ef401b7bd233 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.136159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.174438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.177836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.178123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.178400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.179164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:38.179472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.181138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.199823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 287/565] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:38.199823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28260-eb3f-46c9-8a60-bda1e23a8852 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.200110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 279/566] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:38.200110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.316456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ead083-6b86-4c9d-aeb8-36437b5f290c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.318582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28260-eb3f-46c9-8a60-bda1e23a8852 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.346133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.352692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.353518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.353802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.354079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.357744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.358533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.358815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.359087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.360693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.362191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.370179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.371604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:38.376236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:38.377317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:38.378254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 288/567] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:38.378766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:38.391411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:38.392493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:38.393425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:38.393425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:38.393425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:38.393425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:38.393425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 280/568] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:38.393741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.406952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.487721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c38a4-d17a-477e-aacb-1f6d126bf5c4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.508614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3323-4932-4178-b8d1-aa6eb851737f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.526860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10755268-1bd1-4160-89d6-30be0c830868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.527660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10755268-1bd1-4160-89d6-30be0c830868 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.527944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10755268-1bd1-4160-89d6-30be0c830868 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.528222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10755268-1bd1-4160-89d6-30be0c830868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.538690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10755268-1bd1-4160-89d6-30be0c830868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.540158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10755268-1bd1-4160-89d6-30be0c830868 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.543565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.544357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.544639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.544913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.555648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-10755268-1bd1-4160-89d6-30be0c830868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:38.558148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:38.559117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:38.560024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:38.560024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:38.560024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:38.560024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:38.560024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 289/569] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:38.560821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9b69-d5fb-4d19-a1f3-65106618e39e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.560961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9b69-d5fb-4d19-a1f3-65106618e39e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.561199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9b69-d5fb-4d19-a1f3-65106618e39e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.565190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 290/570] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:38.565322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:38.566227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:38.567121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:38.568366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc9b69-d5fb-4d19-a1f3-65106618e39e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.568528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 281/571] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:38.568528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.602029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.648480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.669589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79141249-08a7-41f5-bac8-b84609106654 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.684071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc9b69-d5fb-4d19-a1f3-65106618e39e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.703585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.704252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.704529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.704802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.705558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:38.705861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.707233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.719024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f36d7-783e-41d9-b4e5-997e0d3f54e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.719199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 282/572] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:38.719693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f36d7-783e-41d9-b4e5-997e0d3f54e1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.719919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.720026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f36d7-783e-41d9-b4e5-997e0d3f54e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.720317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f36d7-783e-41d9-b4e5-997e0d3f54e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.721133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f36d7-783e-41d9-b4e5-997e0d3f54e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:38.721474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f36d7-783e-41d9-b4e5-997e0d3f54e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.722925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f36d7-783e-41d9-b4e5-997e0d3f54e1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.736583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 291/573] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:38.737910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc98d0-b52a-4099-9c0e-623e8367258d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.738203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc98d0-b52a-4099-9c0e-623e8367258d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.738489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc98d0-b52a-4099-9c0e-623e8367258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.755340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7fcc98d0-b52a-4099-9c0e-623e8367258d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:38.756525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7fcc98d0-b52a-4099-9c0e-623e8367258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:38.820288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.828843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 292/574] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:38.829718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc98d0-b52a-4099-9c0e-623e8367258d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.834634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e3d74-b7ae-4ff6-aae2-a5220fe60f6a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.870953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.871609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.871890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.872162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.872878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:38.873204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:38.874569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:38.887600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 283/575] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:38.888464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:38.912084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:38.960385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc98d0-b52a-4099-9c0e-623e8367258d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.997004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:38.997253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e029bd-38f0-47b7-9122-365c39a3ad29 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:38.997718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:38.998002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:38.998411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:38.999196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:38.999502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.000905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.009126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 293/576] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:39.009968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.037319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.038112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.038659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.038659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.046111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.047799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:39.062977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:39.063915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:39.064842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 284/577] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:39.065386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.084474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:39.113828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87a9ad-eddc-404b-9878-e3c16434f69f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.139391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:39.148903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.149584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.149863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.150135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.150957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=0cbe226d7639456ebb94de533de30d28) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:39.151265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.152712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.166258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 294/578] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/role_assignments?user.id=0cbe226d7639456ebb94de533de30d28&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:39.167053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.187628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ac4dc-447d-46b3-9273-761a3ff8eb7f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.234142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.235088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.235380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.235660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.247647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:39.255843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:39.257073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:39.258034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 285/579] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:39.258755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.302936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4394b2-a08c-47f3-a916-798f1e7f26e8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.349294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.349658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.349949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.350228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.361442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:39.362176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.369210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:39.381766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:39.382954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:39.384055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 295/580] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:39.388210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.391590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe901e-5d3d-4f97-9fe1-d9a678143e08 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.427331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.428132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.428417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.428695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.440321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.441838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.443272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:39.460278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:39.462123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:39.463932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:39.463932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:39.463932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:39.463932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:39.466088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 286/581] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:39.466935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.510804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfbae7-824e-4d59-8f1c-cf6be7130319 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.548491 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.549173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.549458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.549731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.550516 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:39.550823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.552281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.565501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 296/582] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:39.566319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.598852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d81f4c-417d-4f43-be91-dd93055fc65d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.632899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:39.646659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.647449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.647801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.648147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.648977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:39.649404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.650889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.660631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 287/583] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:39.661556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.686765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273901f-3248-4a8e-a297-f033a0952e43 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.698828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:39.728496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.729536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.729965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.730384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.731393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:39.731842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.733499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.751755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 297/584] 10.222.0.91 () {56 vars in 1137 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:39.752790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.784873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88625e83-255c-46f2-b898-be13ba5cf854 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.820827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.821533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.821814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.822086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.822812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:39.823117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.824499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.842585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 288/585] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:39.842585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.871162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc08f8-0cb6-466b-a1f7-f505b528f08b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.891863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:39.907777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:39.908681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:39.908970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:39.909290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/f579f2db041f455786a02dcf621619ed/roles/f423841743584b218cefb221f7b3edff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:39.928145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=f579f2db041f455786a02dcf621619ed, role_id=f423841743584b218cefb221f7b3edff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:39.929664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:39.944763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 298/586] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:42:39 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/f579f2db041f455786a02dcf621619ed/roles/f423841743584b218cefb221f7b3edff => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:39.945617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:39.964933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd542b-4aa4-4bb1-9a30-dea75c20ed88 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:39.995872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:40.013481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.014723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.015238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.015767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.029002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.031124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:40.054741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:40.055776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:40.056690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:40.056690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:40.056690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:40.056690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:40.065494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 289/587] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:40.066835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.121406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff9190-6697-4c6f-8b30-63b6047481fa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.172160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.173193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.173456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.173741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.190432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c94369-a6bd-43c5-8ce6-2c1f7cf2adbb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.194435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.195560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:40.196079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:40.219484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:40.220476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:40.221435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:40.221435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:40.221435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:40.221435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:40.222652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 299/588] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:40.227978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.236228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.237255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.237538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.237809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.247872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.249358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:40.264364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:40.265415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:40.266373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 290/589] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:40.266995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.333639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:40.368267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e353-9ba0-4ae7-b181-0049239dadd7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.405818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.407040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.407558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.408061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.408171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140e68f-f322-4993-bbde-cdef7b245491 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.409666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:40.410395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.414160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.431127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 300/590] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:40.432304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.454162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.454872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.455155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.455429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.456161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:40.456468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.457923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.470054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 291/591] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:40.470945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.500799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:40.551239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb74d5-53ce-4376-9e23-5dccc97b46dc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.583299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:40.585337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e191eb-377a-496f-9e69-9cbbd3e808d7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.591894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.593420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.593922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.594761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.596146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:40.596871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.599418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.626298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 301/592] 10.222.0.91 () {56 vars in 1137 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:40.627363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.627932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.628906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.629386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.642021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.644503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:40.665305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:40.666528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:42:40.667705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:40.674187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 292/593] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:40.675200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.756618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94663f-fe3a-4c73-8bbf-3ff526310c7a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.766412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:40.787395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a331-e4fc-479a-81fb-7497769819c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.799932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.800993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.801594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.802025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.810206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.811914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.824066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.825033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.825444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.825822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:40.828857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:40.829938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:40.831048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:40.833806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 302/594] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:40.834768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.841489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.843112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:40.862173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:40.863258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:40.864238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:40.864238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:40.864238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:40.864238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:40.868709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 293/595] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:40.869742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.894314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:40.954288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aef85-6e55-4012-a19d-5eaf174459b2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.977438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a0f60-3a8c-4068-95f7-4c7e05a12dcb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.994748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.995861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.996346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.996831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.997900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:40.998447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.000201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.014753 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.015400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.015685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.015964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.016820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:41.017137 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.019102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.021185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:41.024959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 303/596] 10.222.0.91 () {56 vars in 1139 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:41.026077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.038094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 294/597] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:41.039462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.140416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecaaa2-4241-40b5-86cf-271a131a9e7b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.144657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334f95-a8dc-4957-bc18-737609e16140 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.149954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:41.185602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.185855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.186432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.186740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.187017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.187739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.188254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.188734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.193029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:41.193665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.195508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.201112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.202665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.210103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 304/598] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:41.211329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:41.228628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:41.229745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:41.230684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:41.230684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:41.230684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:41.230684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:41.230684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 295/599] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:41.230684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.283867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:41.346954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7b292-7988-4b24-b7cd-36c3ffc72839 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.347203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73f912-ecd9-4ce6-a7d6-77aed7ef09d6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.384237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.385278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.385479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.385624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.385832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/a0c209755dfc4b4089176a056cf2f683/roles/f423841743584b218cefb221f7b3edff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.386781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.387346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.387770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.395443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.397109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.402442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=a0c209755dfc4b4089176a056cf2f683, role_id=f423841743584b218cefb221f7b3edff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.404032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:41.413670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:41.415078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:41.416113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:41.418335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 296/600] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:41.419399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.424551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 305/601] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:42:41 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/a0c209755dfc4b4089176a056cf2f683/roles/f423841743584b218cefb221f7b3edff => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:41.425240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.461722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:41.544957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05052a3c-3c4e-4db4-9ff3-c17f44a9c2b8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.558872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:41.587480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bdb61-c82d-4c8d-92ba-3d71f03ff001 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.597320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6723e-cfe0-4d6b-bf59-2054565f80e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.598020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6723e-cfe0-4d6b-bf59-2054565f80e3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.598339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6723e-cfe0-4d6b-bf59-2054565f80e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.598619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6723e-cfe0-4d6b-bf59-2054565f80e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.599362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6723e-cfe0-4d6b-bf59-2054565f80e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:41.599668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6723e-cfe0-4d6b-bf59-2054565f80e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.601731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6723e-cfe0-4d6b-bf59-2054565f80e3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.619214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 297/602] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:41.621911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08073-5990-4302-bfc4-5f251fbb7758 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.622419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08073-5990-4302-bfc4-5f251fbb7758 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.622852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08073-5990-4302-bfc4-5f251fbb7758 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.627095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 298/603] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:41.629134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08073-5990-4302-bfc4-5f251fbb7758 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.646616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0520776-cf1e-48f4-9023-a6b915b174d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.647791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0520776-cf1e-48f4-9023-a6b915b174d3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.648137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0520776-cf1e-48f4-9023-a6b915b174d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.648422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0520776-cf1e-48f4-9023-a6b915b174d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.649562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0520776-cf1e-48f4-9023-a6b915b174d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:41.649925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0520776-cf1e-48f4-9023-a6b915b174d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.651771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0520776-cf1e-48f4-9023-a6b915b174d3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.668396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 306/604] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:41.669830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086065d6-45e8-4578-b0fd-79aeecf5a8ac None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.670122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086065d6-45e8-4578-b0fd-79aeecf5a8ac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.670594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086065d6-45e8-4578-b0fd-79aeecf5a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.693677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-086065d6-45e8-4578-b0fd-79aeecf5a8ac None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:41.695151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-086065d6-45e8-4578-b0fd-79aeecf5a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:41.773770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 307/605] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:41.775266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086065d6-45e8-4578-b0fd-79aeecf5a8ac None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.775749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08073-5990-4302-bfc4-5f251fbb7758 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.814167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.814902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.815186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.815460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.823754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.825319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.833645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:41.840981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:41.842004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 299/606] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:41.844355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:41.880459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:41.908327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086065d6-45e8-4578-b0fd-79aeecf5a8ac None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.950655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:41.951593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:41.951878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:41.952158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/8a381dccf1ca4dacaab4e0ae0d473525/roles/f423841743584b218cefb221f7b3edff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:41.969459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=8a381dccf1ca4dacaab4e0ae0d473525, role_id=f423841743584b218cefb221f7b3edff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:41.971208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:41.981236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77ebd7-7637-4106-a306-324094b052d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:41.996142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 308/607] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:42:41 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/8a381dccf1ca4dacaab4e0ae0d473525/roles/f423841743584b218cefb221f7b3edff => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:41.996142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.050073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.051296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.051839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.052323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.068432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:42.081892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:42.094403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:42.095421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:42.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:42.104047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 300/608] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:42.109834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.122644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3998469-0cda-4aac-8206-92b5fa733081 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:42.169624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.170405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.170700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.170986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.171802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:42.172123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:42.175419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:42.192835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 309/609] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:42.199621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.307851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:42.333268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a567c05-1bc5-4318-8062-f6ad1afcfca7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:42.339436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc76ac-903a-4b9e-aca1-f1f57d35f9a8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:42.360754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:42.383823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d866676-582d-4c4d-ba1e-15fde812eb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.384032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9018ba-abdb-4e19-8c81-5eaa373a7d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.384698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9018ba-abdb-4e19-8c81-5eaa373a7d5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.384869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d866676-582d-4c4d-ba1e-15fde812eb4f None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.384982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9018ba-abdb-4e19-8c81-5eaa373a7d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.385274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9018ba-abdb-4e19-8c81-5eaa373a7d5f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.385398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d866676-582d-4c4d-ba1e-15fde812eb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.385633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d866676-582d-4c4d-ba1e-15fde812eb4f None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/79d624b7de50480bb4002eb23ec2b360/roles/f423841743584b218cefb221f7b3edff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.386093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9018ba-abdb-4e19-8c81-5eaa373a7d5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:42.386401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9018ba-abdb-4e19-8c81-5eaa373a7d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:42.387831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9018ba-abdb-4e19-8c81-5eaa373a7d5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:42.399939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 301/610] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:42.401990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d866676-582d-4c4d-ba1e-15fde812eb4f None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=79d624b7de50480bb4002eb23ec2b360, role_id=f423841743584b218cefb221f7b3edff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:42.403665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d866676-582d-4c4d-ba1e-15fde812eb4f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:42.406204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb720a-ef0f-4897-9cf0-aa96a3cdf1c2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.406543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb720a-ef0f-4897-9cf0-aa96a3cdf1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.406830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb720a-ef0f-4897-9cf0-aa96a3cdf1c2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.408937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 302/611] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:42.409795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb720a-ef0f-4897-9cf0-aa96a3cdf1c2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.423989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 310/612] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:42:42 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/79d624b7de50480bb4002eb23ec2b360/roles/f423841743584b218cefb221f7b3edff => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:42:42.425547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f16d-55bc-44b9-be33-753c2b03397f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.425948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f16d-55bc-44b9-be33-753c2b03397f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.426869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f16d-55bc-44b9-be33-753c2b03397f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.445920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6464f16d-55bc-44b9-be33-753c2b03397f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:42.448595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6464f16d-55bc-44b9-be33-753c2b03397f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:42.537860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 311/613] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:42.567196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f16d-55bc-44b9-be33-753c2b03397f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.635025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb720a-ef0f-4897-9cf0-aa96a3cdf1c2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:42.684900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f16d-55bc-44b9-be33-753c2b03397f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:42.687836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:42.721659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.722843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.723905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.724382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.724489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.725461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.725808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.726093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.737811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:42.738580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:42.740813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:42.746927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:42.747882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:42.748752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 312/614] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:42.753214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:42.756014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:42.764791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:42.765776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:42.766710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:42.766710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:42.766710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:42.766710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:42.766710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 303/615] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:42.772655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.868812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94bd0f-59c3-44a9-81af-f5cc794e3216 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:42.905746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.906954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.907445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.907929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.908943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:42.909515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:42.911238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:42.920823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 313/616] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:42.927422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:42.928331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1152c-16e3-4b1b-9825-66de7433ac51 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:42.948529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:42.993369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:42.994052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:42.994336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:42.994624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:42.995363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:42.995672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:42.997191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.019300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 304/617] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:43.026965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.038917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b3bdb-0906-4a7c-b34a-f234655a9acb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:43.075790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:43.076905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:43.077464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:43.077946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:43.089577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:43.091418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:43.109813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:43.110754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:43.111698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:43.111698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:43.111698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:43.111698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:43.113700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 314/618] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:43.119640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.137540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:43.188279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2396b1d-c22d-4a6d-8aa0-ec0ccc4eed4d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:43.224576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:43.225509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:43.225799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:43.226074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:43.241175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:43.241175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:43.256078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:43.257212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:43.258257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:43.258822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 305/619] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:43.265061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:43.271874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.308010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cd921-c65f-4646-ab11-5632fe78a96b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:43.365936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:43.366981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:43.367471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:43.367972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:43.368912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:43.369467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:43.371393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.402244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 315/620] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:43.405412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.477705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4adfef-48b5-4c7c-a015-bf80bbb66833 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:43.501897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:43.522356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:43.523048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:43.523331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:43.523604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:43.524451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:43.524757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:43.526228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.567920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 306/621] 10.222.0.91 () {56 vars in 1135 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:43.578343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.597316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64568533-0c73-4aa4-8d19-dd0e2c313032 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:43.641035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:43.655451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:43.656545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:43.657038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:43.657494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:43.667584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:43.669410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:43.703576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:43.704819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:43.706007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:43.712384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 316/622] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:43.714695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.777793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaed8a-ec91-40a9-ae2e-92c520e5a291 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:43.862452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:43.863278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:43.863559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:43.863872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:43.872732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b67d5-d7a2-46f6-bf3e-4507ae7ff517 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:43.879488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:43.880978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.908798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:43.909945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:43.910596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:43.911772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:43.913240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:43.913766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:43.915417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:43.927968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:43.928924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:43.929916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:43.929916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:43.929916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:43.929916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:43.937646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 307/623] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:43.943773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 317/624] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:43.944588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.950167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:43.998375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.005802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.060819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a471b4-e30d-4a3b-bd5f-e99276ae81de None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.075354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24596bf4-fe2f-404d-8338-a9392f196799 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.099454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.100568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.101214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.101730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.112099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e09ca2-98cc-4953-9f11-ddc25ddcb42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.112775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e09ca2-98cc-4953-9f11-ddc25ddcb42f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.113057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e09ca2-98cc-4953-9f11-ddc25ddcb42f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.113362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e09ca2-98cc-4953-9f11-ddc25ddcb42f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.114128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e09ca2-98cc-4953-9f11-ddc25ddcb42f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:44.114436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e09ca2-98cc-4953-9f11-ddc25ddcb42f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:44.115916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e09ca2-98cc-4953-9f11-ddc25ddcb42f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:44.128714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 308/625] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:44.130557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6acc4-70eb-4bbf-acb0-a4b3bb0c67ef None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.130846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6acc4-70eb-4bbf-acb0-a4b3bb0c67ef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6acc4-70eb-4bbf-acb0-a4b3bb0c67ef None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.132818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 309/626] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:44.134115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6acc4-70eb-4bbf-acb0-a4b3bb0c67ef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:44.200900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:44.202880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:44.243148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.255018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6acc4-70eb-4bbf-acb0-a4b3bb0c67ef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.261005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.294686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f6acb-4fa2-4964-b902-d90ff47b6861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.295494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f6acb-4fa2-4964-b902-d90ff47b6861 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.295835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f6acb-4fa2-4964-b902-d90ff47b6861 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.296286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f6acb-4fa2-4964-b902-d90ff47b6861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.304183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f6acb-4fa2-4964-b902-d90ff47b6861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:44.305681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f6acb-4fa2-4964-b902-d90ff47b6861 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-844f6acb-4fa2-4964-b902-d90ff47b6861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:44.321927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:44.322891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:44.323799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 310/627] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:44.329205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a76f-b500-44c6-8960-c0fe5498a316 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.329205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a76f-b500-44c6-8960-c0fe5498a316 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.329205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a76f-b500-44c6-8960-c0fe5498a316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.329989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 318/628] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:44.331598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:44.345994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e72a76f-b500-44c6-8960-c0fe5498a316 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:44.347183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e72a76f-b500-44c6-8960-c0fe5498a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:44.419974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 311/629] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:44.420939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a76f-b500-44c6-8960-c0fe5498a316 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:44.441595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f96c3c-4e1c-48af-ab4b-67986f2b5ab7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.486246 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.486934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.487223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.487613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.488527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:44.489435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:44.490994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:44.505889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 319/630] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:44.512014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.513607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:44.527962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.551174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a76f-b500-44c6-8960-c0fe5498a316 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.591763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.592802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.593305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.593763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.605126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:44.607175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:44.628200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:44.629191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:44.630685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:44.630685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:44.630685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:44.630685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:44.633021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 312/631] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:44.634039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:44.656373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02488194-8f4a-449c-891c-041f18f28d4f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.697852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.699039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.699554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.700078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.712286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:44.720518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:44.721750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:44.722772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:44.724800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 320/632] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:44.725586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:44.755368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203970d5-87e0-479a-8c55-2053512656e0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.791587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.795676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:44.800064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08189b-a324-4c1a-823b-7b5d68f76ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.800964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08189b-a324-4c1a-823b-7b5d68f76ff7 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.802104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08189b-a324-4c1a-823b-7b5d68f76ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.802104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08189b-a324-4c1a-823b-7b5d68f76ff7 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/0cbe226d7639456ebb94de533de30d28/roles/f423841743584b218cefb221f7b3edff` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.817870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08189b-a324-4c1a-823b-7b5d68f76ff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=0cbe226d7639456ebb94de533de30d28, role_id=f423841743584b218cefb221f7b3edff)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:44.819355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08189b-a324-4c1a-823b-7b5d68f76ff7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:44.833009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 313/633] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:42:44 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/0cbe226d7639456ebb94de533de30d28/roles/f423841743584b218cefb221f7b3edff => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:44.834418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0ca9-6ddf-49ef-8437-bc9164a8f7c4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.834712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0ca9-6ddf-49ef-8437-bc9164a8f7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.835013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0ca9-6ddf-49ef-8437-bc9164a8f7c4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.840715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 314/634] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:44.840715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0ca9-6ddf-49ef-8437-bc9164a8f7c4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:44.847184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1f78f-74fc-4904-9184-e11d13b9ba19 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:44.894536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed7e22-64c7-48d8-aca2-a7424d844578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:44.896784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed7e22-64c7-48d8-aca2-a7424d844578 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.896784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed7e22-64c7-48d8-aca2-a7424d844578 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.896784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed7e22-64c7-48d8-aca2-a7424d844578 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.896784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed7e22-64c7-48d8-aca2-a7424d844578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:44.897664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed7e22-64c7-48d8-aca2-a7424d844578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:44.898691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed7e22-64c7-48d8-aca2-a7424d844578 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:44.919561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 321/635] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:44.921691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8732d3-8966-44fd-a52d-ed52823fe780 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:44.921991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8732d3-8966-44fd-a52d-ed52823fe780 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:44.922323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8732d3-8966-44fd-a52d-ed52823fe780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:44.947635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6e8732d3-8966-44fd-a52d-ed52823fe780 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:44.949274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6e8732d3-8966-44fd-a52d-ed52823fe780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:44.974159 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0ca9-6ddf-49ef-8437-bc9164a8f7c4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.023920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.025017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.025581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.026072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.027024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:45.027518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.029328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.044696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 315/636] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:45.045937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 322/637] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:45.046336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.048371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8732d3-8966-44fd-a52d-ed52823fe780 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.104818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:45.131434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:45.174737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80062d-94c9-45fb-8bb3-36c1866a4ee8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.185031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8732d3-8966-44fd-a52d-ed52823fe780 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.211795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.212709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.213033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.213351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.221027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.222752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.227777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.228576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.228856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.229128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:45.238443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:45.239909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:45.240841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:45.241886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.242046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 316/638] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:45.242046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.246007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:45.269912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:45.271483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:45.272388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:45.272388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:45.272388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:45.272388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:45.276756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 323/639] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:45.277704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.370117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233b141-a591-4c63-8840-02aaeb61d610 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.374229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:45.388492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ee88-d75c-4915-858e-aa155ab09863 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.399989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:45.421484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.422691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.423050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.423329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.428910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25c0d4-67af-4700-9401-72b82f3ebbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.429924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c0d4-67af-4700-9401-72b82f3ebbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.430467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c0d4-67af-4700-9401-72b82f3ebbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.430887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c0d4-67af-4700-9401-72b82f3ebbd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.431907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25c0d4-67af-4700-9401-72b82f3ebbd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:45.432373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25c0d4-67af-4700-9401-72b82f3ebbd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.433765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:45.434138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25c0d4-67af-4700-9401-72b82f3ebbd3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:45.441663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:45.442718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:45.443617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 317/640] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:45.444345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.453190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 324/641] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:45.453190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3f080-b24c-43c4-9b2e-528bd55a2024 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.453190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3f080-b24c-43c4-9b2e-528bd55a2024 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.453190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3f080-b24c-43c4-9b2e-528bd55a2024 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.453791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 325/642] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:45.454483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3f080-b24c-43c4-9b2e-528bd55a2024 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.557790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab572-fbe7-42c8-a4de-7d84ab2e1653 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.573018 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3f080-b24c-43c4-9b2e-528bd55a2024 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.597006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.597868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.598160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.598765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.599780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:45.600092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.602291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.609128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d865c-15e3-499e-a430-8c71cd851964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.610320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d865c-15e3-499e-a430-8c71cd851964 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.610611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d865c-15e3-499e-a430-8c71cd851964 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.610888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d865c-15e3-499e-a430-8c71cd851964 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.612505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d865c-15e3-499e-a430-8c71cd851964 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:45.612505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d865c-15e3-499e-a430-8c71cd851964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.613991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d865c-15e3-499e-a430-8c71cd851964 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.619487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 318/643] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:45.619487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.631759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 326/644] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:45.631759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565a3c7-724b-4cd9-a5d6-98dd1b14c493 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.631759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565a3c7-724b-4cd9-a5d6-98dd1b14c493 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.631759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565a3c7-724b-4cd9-a5d6-98dd1b14c493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.649414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2565a3c7-724b-4cd9-a5d6-98dd1b14c493 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:45.650638 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2565a3c7-724b-4cd9-a5d6-98dd1b14c493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:45.652054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:45.680154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:45.725736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 327/645] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:45.726368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565a3c7-724b-4cd9-a5d6-98dd1b14c493 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.754955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a49e1-9306-42b8-a6b1-1789300bd40f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.804595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.805658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.806121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.806582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.807569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=f579f2db041f455786a02dcf621619ed) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:45.807951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.809737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.817562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 319/646] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/role_assignments?user.id=f579f2db041f455786a02dcf621619ed&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:45.818887 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.856328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565a3c7-724b-4cd9-a5d6-98dd1b14c493 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.913635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.914663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.914979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.915257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:45.928898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:45.931238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:45.934287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:45.945820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a3101-6c2c-4c88-a201-78c9e760f5d1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:45.957146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:45.958247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 13:42:45.959369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:45.962168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 328/647] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:45.963178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:45.991266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:45.992314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:45.992702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:45.993077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.002451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:46.007208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.008865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:46.036790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:46.037936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:46.039021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:46.039021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:46.039021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:46.039021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:46.041479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 320/648] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:46.042438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.102814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253fa9-c2a8-4a14-b1ae-7f83d3dfd8b9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.154029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.155095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.155540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.155967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.168008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:46.177908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c7cbf-d96e-4598-8ec6-fa6d9384398c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:46.178079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:46.179551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 329/649] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:46.180425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.227062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.228094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.228567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.229029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.230544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:46.231064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.233409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.245378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:46.263954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 321/650] 10.222.0.91 () {56 vars in 1137 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:46.263954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.297783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:46.310515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7871bf-94ec-4559-97de-ac6b347467b9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.351721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.352799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.353352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.353912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.355623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:46.356218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.357980 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.370104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90ef53-6405-4ea3-bc5d-bda3cf6909cd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.382547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 330/651] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:46.383886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.405851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.406701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.406983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.407251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.407994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:46.408298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.409700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.419851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 322/652] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:46.420686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.483006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:46.500375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a773a-54d4-46bb-b9d7-3e6987f15d43 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.526917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4753-4fae-498e-a2e4-1e7e6b854e53 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.538241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.539311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.539807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.540307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.563737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.564538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.564893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.565185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.574528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.576273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.577707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:46.599401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:46.601020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:42:46.602044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:46.604376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 323/653] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:46.605484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.631093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.632863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.719884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d7d48-5d23-413d-aa00-2e0b177f0858 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.749031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:46.765118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 331/654] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:46.766397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.768653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.769662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.770591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.771040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.783226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.784951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:46.804812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:46.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:46.807172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:46.807172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:46.807172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:46.807172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:46.813486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 324/655] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:46.814602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.821611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:46.885810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e95bd-a28b-4b29-9804-db15c3868b53 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.925384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110f97f-8b99-47c4-82e9-6afcab90c169 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:46.925829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.926703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.927291 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.927720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.928861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:46.929384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.930987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:46.955271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 332/656] 10.222.0.91 () {56 vars in 1137 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:46.956648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:46.963381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:46.964221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:46.964604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:46.964981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:46.979182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:46.980738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:47.001356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:47.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:47.003562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:47.003562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:47.003562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:47.003562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:47.007907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 325/657] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:47.008859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.037785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.080712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.084892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a92f-5f4e-4583-9d81-73c84fd4a8c4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.122604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.123519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4de7-ae93-49f6-a6c4-30e06fdf779d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.123886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.124401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.124979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.126052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:47.126598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.128236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.143068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 333/658] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:47.144364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.167981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f21fb-fa3f-42ab-a4cd-cf4f0d5bfa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.168798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f21fb-fa3f-42ab-a4cd-cf4f0d5bfa4d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.169082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f21fb-fa3f-42ab-a4cd-cf4f0d5bfa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.169393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f21fb-fa3f-42ab-a4cd-cf4f0d5bfa4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.247517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f21fb-fa3f-42ab-a4cd-cf4f0d5bfa4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.248884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f21fb-fa3f-42ab-a4cd-cf4f0d5bfa4d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.261991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8eaa0-6355-4ce1-8fd6-310a5467dfc2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.275695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.298057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.298730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.299017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.299296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.300022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:47.300327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.301763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.313632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 334/659] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:47.314441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.338050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.365900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 326/660] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:47.367593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54517-766c-4c4c-9dc9-37d4c8e8a43a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.367879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54517-766c-4c4c-9dc9-37d4c8e8a43a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.368153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54517-766c-4c4c-9dc9-37d4c8e8a43a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.369614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 327/661] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:47.374208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54517-766c-4c4c-9dc9-37d4c8e8a43a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.423729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6ceed-93d4-4b97-83cb-b040e0e6d407 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.463375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.464188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.464475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.464757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.472700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.474218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.480912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54517-766c-4c4c-9dc9-37d4c8e8a43a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:47.489617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:47.490591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 335/662] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:47.491570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.510161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.522553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513e170-7803-47ef-adc9-b76bae62727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.523352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e170-7803-47ef-adc9-b76bae62727c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.523643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e170-7803-47ef-adc9-b76bae62727c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.523919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513e170-7803-47ef-adc9-b76bae62727c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.533916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e170-7803-47ef-adc9-b76bae62727c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.535382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513e170-7803-47ef-adc9-b76bae62727c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b513e170-7803-47ef-adc9-b76bae62727c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:47.549834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:47.551016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:47.552501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 328/663] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:47.553011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b98dfb-f629-4840-bd7a-7552ffa547b1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.553090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b98dfb-f629-4840-bd7a-7552ffa547b1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.553596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b98dfb-f629-4840-bd7a-7552ffa547b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.571814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6b98dfb-f629-4840-bd7a-7552ffa547b1 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:47.573007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6b98dfb-f629-4840-bd7a-7552ffa547b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:47.589550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.609833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff133b36-72aa-4ca4-8019-0133e41eaeaa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.649395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 329/664] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:47.649395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b98dfb-f629-4840-bd7a-7552ffa547b1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.651688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.652440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.653520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.654042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.667892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.669556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:47.688683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:47.689736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:47.696431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:47.696431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:47.696431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:47.696431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:47.696431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 336/665] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:47.696431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.763435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b98dfb-f629-4840-bd7a-7552ffa547b1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.781480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.808204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.808974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.809099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a5014-e4d2-4242-9fa2-378b521cf83c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.809380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.809514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.810264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:47.810754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.812173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.830937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 330/666] 10.222.0.91 () {56 vars in 1139 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:47.831866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.846875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:47.853897 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.854578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.854865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.855145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.855949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:47.856261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.857796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.871562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 337/667] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:47.872402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.946946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6971949-064d-40fc-9429-6004c783c3d1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.983523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a3a08-3298-4c54-a218-e0049856fd95 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.985830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.986627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.986907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.987178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.998339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:48.006211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:48.007362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:48.008490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:48.012094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 331/668] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:48.012944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.026380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.026987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.027277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.027569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.028283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:48.028599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.030019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.043675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 338/669] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:48.044468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.052291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:48.115624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:48.126651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bf302-6472-4729-9f2b-8af5050796e9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.163829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.164485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.164788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.165048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.169227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa675f2-1b0a-403e-80aa-a739edc3fdda None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.208550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.209406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.209687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.209958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.210797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=a0c209755dfc4b4089176a056cf2f683) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:48.211108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.212621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.218660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 339/670] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/role_assignments?user.id=a0c209755dfc4b4089176a056cf2f683&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:48.226227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.248425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.249980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.287037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:48.354529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb6194-d603-4f90-b8b4-c0eac9e02ebd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.373690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:48.378910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 332/671] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:48.378910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.400538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9c650-87f5-4562-8d44-4fc9e9124d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.401441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9c650-87f5-4562-8d44-4fc9e9124d9c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.401810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9c650-87f5-4562-8d44-4fc9e9124d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.402093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9c650-87f5-4562-8d44-4fc9e9124d9c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.402914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9c650-87f5-4562-8d44-4fc9e9124d9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:48.403160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9c650-87f5-4562-8d44-4fc9e9124d9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.405382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9c650-87f5-4562-8d44-4fc9e9124d9c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.417226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 340/672] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:48.417226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed08fb-4fcb-442b-98ef-569df743491b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.418304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed08fb-4fcb-442b-98ef-569df743491b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.418591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed08fb-4fcb-442b-98ef-569df743491b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.420131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 341/673] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:48.420863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed08fb-4fcb-442b-98ef-569df743491b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.486403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a5083-0617-4106-957c-32e4e1186c28 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.518121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:48.523507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.524700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.525049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.525361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.532448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.533946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:48.548626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:48.549720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:48.550728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 333/674] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:48.551843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.558968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed08fb-4fcb-442b-98ef-569df743491b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.609035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3facde0-78ad-46fb-aaad-bdf3af0e974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.609972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3facde0-78ad-46fb-aaad-bdf3af0e974d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.610316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3facde0-78ad-46fb-aaad-bdf3af0e974d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.610654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3facde0-78ad-46fb-aaad-bdf3af0e974d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.624383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3facde0-78ad-46fb-aaad-bdf3af0e974d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.626003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3facde0-78ad-46fb-aaad-bdf3af0e974d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d3facde0-78ad-46fb-aaad-bdf3af0e974d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:48.645218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:48.646199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:48.647093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:48.647093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:48.647093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:48.647093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:48.647093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 342/675] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:48.648020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a42e-540b-4743-9c76-4a145d6a3f72 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.648375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a42e-540b-4743-9c76-4a145d6a3f72 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.648656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a42e-540b-4743-9c76-4a145d6a3f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.666031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9e20a42e-540b-4743-9c76-4a145d6a3f72 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:48.667218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e20a42e-540b-4743-9c76-4a145d6a3f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:48.668401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525bd86-71de-467c-8c9b-9ec5f3648e00 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.694097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:48.703234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.704039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.704323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.704597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.715357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.717280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:48.734922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:48.735884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:48.736787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:48.736787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:48.736787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:48.736787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:48.736787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 334/676] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:48.737063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.746252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 343/677] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:48.750168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20a42e-540b-4743-9c76-4a145d6a3f72 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.797669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:48.855613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20a42e-540b-4743-9c76-4a145d6a3f72 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.859190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c55c8-e39e-4b87-980e-ebc5dd9ad501 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:48.889378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.890046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.890325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.890602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.891326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:48.891630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.893015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.896418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:48.897342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.897807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.898265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.899244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:48.899731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:48.901400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:48.905003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 344/678] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:48.905894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.916336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 335/679] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:48.917199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:48.934890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.017237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ddcb9-2fe3-4c8b-a77d-641caa0fdc53 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.027429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c7a54-7e70-4448-b988-690156814108 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.030109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.055751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.056427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.056712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.056991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.057747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:49.058056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.059436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.065289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.066274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.067207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.067489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.075876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 345/680] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:49.075876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.081470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:49.089907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:49.091107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:49.091987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 336/681] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:49.092696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.171628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.191855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80a39c-b12a-467d-b004-ff928d51eb72 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.219200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ef86-3fa4-4d5c-8cdf-401f5f88fcdc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.227007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.227810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.228099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.228380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.235509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.236979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:49.251898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:49.252863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:49.253816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 346/682] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:49.254963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.264461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.265124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.265436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.265709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.266546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=8a381dccf1ca4dacaab4e0ae0d473525) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:49.266855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.268711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.277007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 337/683] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/role_assignments?user.id=8a381dccf1ca4dacaab4e0ae0d473525&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:49.286578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.323691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.388776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897a123-ff87-40dc-bcc7-77fa8cd0e992 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.435303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebec09-0a96-4b88-9f76-5185f47ce8d3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.436340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.437344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.437725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.438191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.448059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.449869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.453936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:49.469404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:49.470647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:49.471632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 347/684] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:49.472863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.482569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.483261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.483545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.483825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.484653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:49.484888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.486341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.496313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 338/685] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:49.497025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.593263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f564310-b179-4017-ab1d-108b194aa854 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.618636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.631273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5fdc-43a2-45d4-b768-60a1a6aaf743 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.636923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.637628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.637913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.638191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.638989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:49.639300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.641322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.654626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 348/686] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:49.656160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.675646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.677279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.677790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.678234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.679282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:49.679823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.681525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.702847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 339/687] 10.222.0.91 () {56 vars in 1135 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:49.703637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.741277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.787361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e5001-0288-4592-9973-bd93948f9391 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.814236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a194f-db62-4c21-922d-f1623708da24 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.824761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.825708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.825991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.826262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.838279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.839989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.850614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.851272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.851550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.851821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.852643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=79d624b7de50480bb4002eb23ec2b360) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:49.852945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.854400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:49.859282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:49.860171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:49.861083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:49.861083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:49.861083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:49.861083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:49.861331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 340/689] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/role_assignments?user.id=79d624b7de50480bb4002eb23ec2b360&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:49.862233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.865466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 349/689] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:49.866416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.877889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:49.979531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5345e5-ce80-4044-b18e-0dc4e481eea9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.986642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabde235-2b49-4ba1-a704-32826ddcf1ad None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:50.002844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:50.022626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.023561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.023942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.024314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.028282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.029018 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.029411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.029693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.030515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:50.030821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.032251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.045882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 350/690] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:50.051839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:50.102609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.104251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.120885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:50.170387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743616c4-25ad-4062-9d42-ee335d2926c3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:50.213657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.214450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.214779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.215076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.224379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.225877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:50.243465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:50.244472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:50.245535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 351/691] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:50.250312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:50.275899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 341/692] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:50.281322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:50.290158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:50.365849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3d90d-2fe7-4c84-bacb-d5b3a133cdaf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:50.402709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2e26-782d-4847-9e5d-5510e2beeeff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:50.403978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.404755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.405042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.405360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.406459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:50.406774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.408274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.421820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:50.430316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 352/693] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:50.436053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:50.447653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.448930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.449441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.449941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.462211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.464149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:50.492268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:50.493594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:50.494529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:50.494529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:50.494529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:50.494529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:50.497010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 342/694] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:50.504415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:50.547777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:50.558384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56015f58-06e0-4feb-9c17-cab0b5065344 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:50.615145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bebdf7-45aa-43b5-a66b-a0f73a6ffd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.615805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bebdf7-45aa-43b5-a66b-a0f73a6ffd95 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.616085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bebdf7-45aa-43b5-a66b-a0f73a6ffd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.616357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bebdf7-45aa-43b5-a66b-a0f73a6ffd95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.665308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916ec92-30cb-44e0-a5b7-3472984adae6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:50.702074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bebdf7-45aa-43b5-a66b-a0f73a6ffd95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.702638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec2666-2301-4f3d-9fa6-349698c92d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.702829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2666-2301-4f3d-9fa6-349698c92d67 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.703265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2666-2301-4f3d-9fa6-349698c92d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.703546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bebdf7-45aa-43b5-a66b-a0f73a6ffd95 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.703662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2666-2301-4f3d-9fa6-349698c92d67 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.704412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2666-2301-4f3d-9fa6-349698c92d67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:50.704792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2666-2301-4f3d-9fa6-349698c92d67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.706274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2666-2301-4f3d-9fa6-349698c92d67 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.719807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 343/695] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:50.721437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab37f35-4cc0-49a8-b694-1fd37d5a55c2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.721810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab37f35-4cc0-49a8-b694-1fd37d5a55c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.722186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab37f35-4cc0-49a8-b694-1fd37d5a55c2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.728503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 344/696] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:50.729636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab37f35-4cc0-49a8-b694-1fd37d5a55c2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:50.758534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:50.835701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:50.850140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 353/697] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:50.851492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd153-aae6-4879-938b-62aef7c899a2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.851778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd153-aae6-4879-938b-62aef7c899a2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.852059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd153-aae6-4879-938b-62aef7c899a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.856476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab37f35-4cc0-49a8-b694-1fd37d5a55c2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:50.873234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-458fd153-aae6-4879-938b-62aef7c899a2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:50.874518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-458fd153-aae6-4879-938b-62aef7c899a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:50.901831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:50.902686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:50.902971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:50.903242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:50.911936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:50.913544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:50.930623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:50.932680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:50.934783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:50.937198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 345/698] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:50.941262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:50.956054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 354/699] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:50.957003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fd153-aae6-4879-938b-62aef7c899a2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.054883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:51.077706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b02f6-c739-4bd3-bca5-8ab5fb33700d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.087777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fd153-aae6-4879-938b-62aef7c899a2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.114008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:51.130171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.131453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.131979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.132486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.135020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.135685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.136260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.136569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.137391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:51.137702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.139345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.146765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.149046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.158835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 355/700] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:51.158835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:51.177496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:51.178490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:51.179401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:51.179401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:51.179401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:51.179401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:51.183858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 346/701] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:51.184904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.284030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a2522-aa01-4486-9d4f-80e4d2c3945c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.292470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e4413-aba7-4154-929a-30cd1842d22e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.322411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.323642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.324193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.324644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.328780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.329483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.329762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.330033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.330864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=0cbe226d7639456ebb94de533de30d28) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:51.331170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.332640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.335661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:51.345200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 347/702] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:51 2022] GET /identity/v3/role_assignments?user.id=0cbe226d7639456ebb94de533de30d28&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:51.345200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.345484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.345484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:51.381637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:51.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:51.383521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:51.383521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:51.383521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:51.383521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:51.393852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 356/703] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:51.394849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.475188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:51.485211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb523e-5866-4b44-8d85-b90ea6dcbb27 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.530041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.530746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.531032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.531310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.532076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:51.532364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.542066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.579665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 348/704] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:51.585544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8175ca-0fd8-4b15-90a7-a3d43779a430 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.586201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.626644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.627671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.628165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.628680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.629874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:51.630425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.632093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.646295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 357/705] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:51.654141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.681538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:51.720249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c03541-d132-4f2a-bc57-64c77c0e4bfe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.757121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:51.760551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.762626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.762981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.763394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.768943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6257a7-81b5-4b64-a6cb-c9b587115ad2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.773439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.775038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:51.791196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:51.792379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:51.793439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:51.793902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 349/706] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:51.799590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:51.814434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.815526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.816030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.816545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.817595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.819302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:51.849861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 358/707] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:42:51 2022] POST /identity/v3/users => generated 254 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:51.945996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91439866-8a54-49b4-83b0-9176959306bd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:51.971848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:51.990457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bac9a-3a4f-4c11-8f3a-2c0b887fb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:51.991128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bac9a-3a4f-4c11-8f3a-2c0b887fb03d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:51.991409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bac9a-3a4f-4c11-8f3a-2c0b887fb03d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:51.991679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bac9a-3a4f-4c11-8f3a-2c0b887fb03d None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:51.992531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bac9a-3a4f-4c11-8f3a-2c0b887fb03d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:51.992839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bac9a-3a4f-4c11-8f3a-2c0b887fb03d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:51.994396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bac9a-3a4f-4c11-8f3a-2c0b887fb03d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:52.010238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 350/708] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:42:52.016541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:52.026808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23248-9b9f-40d2-a47d-5ef556697757 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.027455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23248-9b9f-40d2-a47d-5ef556697757 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.027977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23248-9b9f-40d2-a47d-5ef556697757 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.029896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 351/709] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:52.038480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf65736-aedb-4bfe-8646-de8818cf3f89 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.039079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf65736-aedb-4bfe-8646-de8818cf3f89 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.039632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf65736-aedb-4bfe-8646-de8818cf3f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.068828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6cf65736-aedb-4bfe-8646-de8818cf3f89 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:52.068828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6cf65736-aedb-4bfe-8646-de8818cf3f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:52.178910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 352/710] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:52.182286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ddc5-dab6-4aad-a5a4-e030e75d5a48 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:52.200249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf65736-aedb-4bfe-8646-de8818cf3f89 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:52.234938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:52.236439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.237341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.237890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.239225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=f579f2db041f455786a02dcf621619ed) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:52.240525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:52.242238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:52.250366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 359/711] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:52 2022] GET /identity/v3/role_assignments?user.id=f579f2db041f455786a02dcf621619ed&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:52.329322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf65736-aedb-4bfe-8646-de8818cf3f89 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:52.343455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:52.375837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:52.376681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.376969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.377281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.377955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:52.379898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:52.396600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:52.397687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:52.398608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 353/712] 10.222.0.91 () {56 vars in 1129 bytes} [Thu May 26 13:42:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:42:52.403929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:52.437259 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:52.557063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb036b6c-8720-455e-bc8d-8c69d29a186a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:52.600677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f6c4b-a022-453c-8f9a-5ca2223303d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:52.601396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f6c4b-a022-453c-8f9a-5ca2223303d8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.601678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f6c4b-a022-453c-8f9a-5ca2223303d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.601943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f6c4b-a022-453c-8f9a-5ca2223303d8 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.602660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f6c4b-a022-453c-8f9a-5ca2223303d8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:52.602964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f6c4b-a022-453c-8f9a-5ca2223303d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:52.604352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f6c4b-a022-453c-8f9a-5ca2223303d8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:52.609643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 360/713] 10.222.0.91 () {56 vars in 1124 bytes} [Thu May 26 13:42:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:52.615733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:52.739054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc575383-8290-40c7-a33a-6885f604b203 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.739298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc575383-8290-40c7-a33a-6885f604b203 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.739533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc575383-8290-40c7-a33a-6885f604b203 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.741030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 361/714] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:52.746070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ac138-5323-445e-a387-63f23989923f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.747236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ac138-5323-445e-a387-63f23989923f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.747849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ac138-5323-445e-a387-63f23989923f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.776526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8a0ac138-5323-445e-a387-63f23989923f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:52.779799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8a0ac138-5323-445e-a387-63f23989923f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:52.786821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b9eeb-5b7f-4844-a926-fb56bafdba3e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:52.847394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:52.848395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:52.848852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:52.849327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:52.850229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:52.850709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:52.852247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:52.858720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 354/715] 10.222.0.91 () {56 vars in 1124 bytes} [Thu May 26 13:42:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:52.914391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 362/716] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:52.959842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:53.016335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:53.100424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305ecb9-6d90-4813-8de8-d7798cfad81f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:53.137418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:53.138431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:53.138710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:53.138983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:53.139647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:53.141015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:53.153617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:53.154703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:53.155566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 355/717] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:52 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:53.161732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ac138-5323-445e-a387-63f23989923f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:53.179237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:53.290346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ac138-5323-445e-a387-63f23989923f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:53.334910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d6374-f043-455d-82e3-f9a09c5afd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:53.335607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6374-f043-455d-82e3-f9a09c5afd0b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:53.335889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6374-f043-455d-82e3-f9a09c5afd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:53.336161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6374-f043-455d-82e3-f9a09c5afd0b None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:53.336874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d6374-f043-455d-82e3-f9a09c5afd0b None admin] RBAC: Adding query filter params (name=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:53.337193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d6374-f043-455d-82e3-f9a09c5afd0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:53.338579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d6374-f043-455d-82e3-f9a09c5afd0b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:53.343888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 363/718] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:42:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:53.350106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:53.488608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fffe40-956e-4085-b6de-9c3a9c2cc6c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:53.490194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:53.524486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5e0e2-78cd-4502-b0f6-66dfebbe8f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:53.525178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5e0e2-78cd-4502-b0f6-66dfebbe8f70 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:53.525471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5e0e2-78cd-4502-b0f6-66dfebbe8f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:53.525745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5e0e2-78cd-4502-b0f6-66dfebbe8f70 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:53.526457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5e0e2-78cd-4502-b0f6-66dfebbe8f70 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:53.526765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5e0e2-78cd-4502-b0f6-66dfebbe8f70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:53.528134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5e0e2-78cd-4502-b0f6-66dfebbe8f70 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:53.537273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 356/719] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:42:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:54.539214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc400e-3199-42f5-bdfd-6eda06f6e432 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:54.540194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc400e-3199-42f5-bdfd-6eda06f6e432 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:54.540641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc400e-3199-42f5-bdfd-6eda06f6e432 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:54.542677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 364/720] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:54.549500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b7e1f-1f66-4190-b8d1-60401b278f25 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:54.550462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b7e1f-1f66-4190-b8d1-60401b278f25 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:54.551014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b7e1f-1f66-4190-b8d1-60401b278f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:54.568915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5f5b7e1f-1f66-4190-b8d1-60401b278f25 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:54.570481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f5b7e1f-1f66-4190-b8d1-60401b278f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:54.645763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 357/721] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:54.667131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc400e-3199-42f5-bdfd-6eda06f6e432 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:54.770820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:54.817889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc400e-3199-42f5-bdfd-6eda06f6e432 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:54.861909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10de68-40fd-4cc2-a0cd-05e2db47d4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:54.863087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10de68-40fd-4cc2-a0cd-05e2db47d4ad None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:54.863600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10de68-40fd-4cc2-a0cd-05e2db47d4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:54.864068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10de68-40fd-4cc2-a0cd-05e2db47d4ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:54.883919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10de68-40fd-4cc2-a0cd-05e2db47d4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bc10de68-40fd-4cc2-a0cd-05e2db47d4ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:54.892064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:54.892989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:54.894004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:54.897242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 365/722] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:54.905180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b7e1f-1f66-4190-b8d1-60401b278f25 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:54.933794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a3091-15ad-416b-ab79-7df0e055bd83 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:54.934413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a3091-15ad-416b-ab79-7df0e055bd83 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:54.935040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a3091-15ad-416b-ab79-7df0e055bd83 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:54.937192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 366/723] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:54.942201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e9194-7220-4432-92b0-0dfe69dd0f15 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:54.942673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e9194-7220-4432-92b0-0dfe69dd0f15 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:54.943093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e9194-7220-4432-92b0-0dfe69dd0f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:54.965235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-729e9194-7220-4432-92b0-0dfe69dd0f15 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:54.966419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-729e9194-7220-4432-92b0-0dfe69dd0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:55.041803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 367/724] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:42:55.043770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e218-7d4b-461f-ae17-2970dd76fb4e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.044268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e218-7d4b-461f-ae17-2970dd76fb4e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.044763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e218-7d4b-461f-ae17-2970dd76fb4e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.046738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.048096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 368/725] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:55.053083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6cbdd-63cd-49a9-acb5-b985ec953cef None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.053391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6cbdd-63cd-49a9-acb5-b985ec953cef None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.053839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6cbdd-63cd-49a9-acb5-b985ec953cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.069947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b7e1f-1f66-4190-b8d1-60401b278f25 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.084080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-98d6cbdd-63cd-49a9-acb5-b985ec953cef None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:55.085564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-98d6cbdd-63cd-49a9-acb5-b985ec953cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:55.107779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.108538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.108889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.109260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.110059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:55.110587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.112239 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.123481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 358/726] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:55.124792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.154690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.168868 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 369/727] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:55.170125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6cbdd-63cd-49a9-acb5-b985ec953cef None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.260993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e996e4-96a8-4b1a-8c4f-7e08e86f8464 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.283807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6cbdd-63cd-49a9-acb5-b985ec953cef None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.305662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.306471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.306748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.307025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.322006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:55.323764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.324818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.325404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.325914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:55.330305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:55.331298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:55.332709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 359/728] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:55.333464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.344869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.347529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.361477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:55.370625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:55.371566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:55.372462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:55.372462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:55.372462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:55.372462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:55.380220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 370/729] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:55.380220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.425944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.482491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5f85-ffe2-4fab-8177-9c40865db929 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.489599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e7aa2-8464-4a88-91cd-93b90ad81916 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.522906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.524739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.525264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.526018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.534793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.535555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.535842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.536112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.536868 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:55.537188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.538582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.541747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:42:55.552738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 371/730] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:55.552738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:42:55.556146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:55.557372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:42:55.558266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:55.558947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 360/731] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:55.560450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.677592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dc84c-f85e-4024-8472-d35063078994 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.680759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.684294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.714194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1014cee-828d-457e-8806-936fa6b7c362 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.720748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a8628-d136-403e-96b7-678367100b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.721790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a8628-d136-403e-96b7-678367100b42 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.722219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a8628-d136-403e-96b7-678367100b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.722788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a8628-d136-403e-96b7-678367100b42 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.723780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a8628-d136-403e-96b7-678367100b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:55.724332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a8628-d136-403e-96b7-678367100b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.726085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a8628-d136-403e-96b7-678367100b42 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.740497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 372/732] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:55.741702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a8628-d136-403e-96b7-678367100b42 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.762207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.763433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.763912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.764380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.775898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.777661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:55.796640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:55.797927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:55.798854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:55.798854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:55.798854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:55.798854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:55.801403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 361/733] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:55.802474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.859755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a8628-d136-403e-96b7-678367100b42 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.896093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543eb79a-edea-46c3-b5c0-c7fb85eed5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.896768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb79a-edea-46c3-b5c0-c7fb85eed5ab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.897053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb79a-edea-46c3-b5c0-c7fb85eed5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.897367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb79a-edea-46c3-b5c0-c7fb85eed5ab None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.898117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543eb79a-edea-46c3-b5c0-c7fb85eed5ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:55.898424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543eb79a-edea-46c3-b5c0-c7fb85eed5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.899800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543eb79a-edea-46c3-b5c0-c7fb85eed5ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.908380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 373/734] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:55.909714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636737a-80ce-4c39-8719-795849a537f9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.910001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636737a-80ce-4c39-8719-795849a537f9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.910315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636737a-80ce-4c39-8719-795849a537f9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.915518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 374/735] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:55.915518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636737a-80ce-4c39-8719-795849a537f9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.918734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc400f-0385-4dc3-9b04-c15fda990d5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.949503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.955726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:55.962468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.963267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.963544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.963815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.972178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.974152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:55.988357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:55.990202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:42:55.991195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:55.994756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 362/736] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:55.995373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.034025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636737a-80ce-4c39-8719-795849a537f9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.076124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ddc0c-b782-4d56-a691-8a80998c6c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.076991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ddc0c-b782-4d56-a691-8a80998c6c79 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.077106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ddc0c-b782-4d56-a691-8a80998c6c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.077382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ddc0c-b782-4d56-a691-8a80998c6c79 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.078129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ddc0c-b782-4d56-a691-8a80998c6c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:56.078444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ddc0c-b782-4d56-a691-8a80998c6c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.079883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ddc0c-b782-4d56-a691-8a80998c6c79 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.095412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 375/737] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:56.101856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f674419-b5fe-46b8-bec9-cfede82ae8cc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.102175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f674419-b5fe-46b8-bec9-cfede82ae8cc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.102426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f674419-b5fe-46b8-bec9-cfede82ae8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.106724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16afe3-0157-44e4-8363-173723eba6db None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.145818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.146646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.146933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.147205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.159636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.161655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:56.181682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:56.182642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:56.183549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:56.183549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:56.183549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:56.183549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:56.183549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 363/738] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:56.184264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.234307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:56.247440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7f674419-b5fe-46b8-bec9-cfede82ae8cc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:56.248643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7f674419-b5fe-46b8-bec9-cfede82ae8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:56.296479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daaa19-3a70-4c1f-bab0-8feacfaf8f09 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.317424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 376/739] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:56.318338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f674419-b5fe-46b8-bec9-cfede82ae8cc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.330049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.330724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.331011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.331285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.332646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:56.332955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.334412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.347036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:56.352901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 364/740] 10.222.0.91 () {56 vars in 1135 bytes} [Thu May 26 13:42:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:56.353847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.430756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f674419-b5fe-46b8-bec9-cfede82ae8cc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.448960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:56.452707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98177328-ceb1-4cdb-b7e2-f2b795f12b94 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.464075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f02b4-45bc-4438-bcc0-2fcc8c117d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.464870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f02b4-45bc-4438-bcc0-2fcc8c117d3b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.465169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f02b4-45bc-4438-bcc0-2fcc8c117d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.465447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f02b4-45bc-4438-bcc0-2fcc8c117d3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.473946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f02b4-45bc-4438-bcc0-2fcc8c117d3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.475409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f02b4-45bc-4438-bcc0-2fcc8c117d3b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.489081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0b3f02b4-45bc-4438-bcc0-2fcc8c117d3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:56.489684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:56.490597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:42:56.491567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:56.492484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.492484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.492484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.492676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 377/741] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:42:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:56.492744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:56.492744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.492873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd064d2-1118-4d69-9256-a62b5c6cb5b0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.492873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd064d2-1118-4d69-9256-a62b5c6cb5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.492995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.493061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd064d2-1118-4d69-9256-a62b5c6cb5b0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.494136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 378/742] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:56.494830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd064d2-1118-4d69-9256-a62b5c6cb5b0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.506644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 365/743] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:56.507475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.572545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:56.596110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd064d2-1118-4d69-9256-a62b5c6cb5b0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.609303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d153f-a9a3-4988-95ce-26d8776489a0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.630880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.631550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.631851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.632118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.632835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.634278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.644449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5298e3-5fb9-4592-83e1-00df4b9ea85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.645178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5298e3-5fb9-4592-83e1-00df4b9ea85a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.645476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5298e3-5fb9-4592-83e1-00df4b9ea85a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.645805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5298e3-5fb9-4592-83e1-00df4b9ea85a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.648676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 379/744] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:42:56 2022] POST /identity/v3/services => generated 251 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:56.649434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.688172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:56.741790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5298e3-5fb9-4592-83e1-00df4b9ea85a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.743357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5298e3-5fb9-4592-83e1-00df4b9ea85a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.790655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0193-310c-4c1f-aa2f-b889a627fe7a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:56.824264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:56.831361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:56.832088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.832572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.832856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.833759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:56.834069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:56.836294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:56.860531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 366/745] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:42:56.860788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 380/746] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:42:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:56.861660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:56.861857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a45c0-bdc6-45ad-9b87-8b60bd771a38 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:56.862099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a45c0-bdc6-45ad-9b87-8b60bd771a38 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:56.862382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a45c0-bdc6-45ad-9b87-8b60bd771a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:56.885144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f73a45c0-bdc6-45ad-9b87-8b60bd771a38 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:56.886681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f73a45c0-bdc6-45ad-9b87-8b60bd771a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:56.984315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3501e7f-f7f3-4092-b924-b8d0db6158ab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.022588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.023341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.023623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.023899 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.032778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.034278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:42:57.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:57.052605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:42:57.053628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:57.057505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 381/747] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:42:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:57.058132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.071549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:57.085961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 367/748] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:42:57.087166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a45c0-bdc6-45ad-9b87-8b60bd771a38 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.126461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:57.179803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9e94b-9db7-4432-994b-280756e8697e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.219303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a45c0-bdc6-45ad-9b87-8b60bd771a38 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.220506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.221349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.221634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.221919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.232665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.234175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:57.258012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:57.258955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:57.259857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:57.259857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:57.259857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:57.259857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:57.259857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 382/749] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:57.260678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.271316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.272172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.272457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.272787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.287786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.289887 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:57.314187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:57.315131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:57.316032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:57.316032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:57.316032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:57.316032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:57.317301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 368/750] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:57.318298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.376313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:57.384049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22883d9-725d-40c6-aefe-41962f0613fd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.385901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:57.422029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.422966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.423249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.423525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.424363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:57.424669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.426167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:57.445481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 383/751] 10.222.0.91 () {56 vars in 1149 bytes} [Thu May 26 13:42:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:57.446320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.457838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9630561-b80a-4cb1-b96c-bb168c6cdac9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.503371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.504627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.504912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.505206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.505964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.507329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:57.528772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 369/752] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:42:57 2022] POST /identity/v3/services => generated 240 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:42:57.529968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.575812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea6211-58ae-4f4b-ba8a-087680980874 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.637946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fc060-6805-4809-a555-081058f6474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.638628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fc060-6805-4809-a555-081058f6474e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.638911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fc060-6805-4809-a555-081058f6474e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.639187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fc060-6805-4809-a555-081058f6474e None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.639929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fc060-6805-4809-a555-081058f6474e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:57.640238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fc060-6805-4809-a555-081058f6474e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.641771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fc060-6805-4809-a555-081058f6474e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:57.670268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 384/753] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:57.670268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fc060-6805-4809-a555-081058f6474e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.700002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:57.769003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbc885-6486-436b-8672-50c75d5d0a7e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.771517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:57.815928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.817051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.817378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.817658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.818406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:57.818715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.820094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:57.834579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 370/754] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:57.835550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:57.846062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fc060-6805-4809-a555-081058f6474e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.892810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.893775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.894065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.894341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.953812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675a16-85f8-4256-8300-3154483764a3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:57.990294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:57.991210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:57.991627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:57.992086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:57.992531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:57.992682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.001967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.003714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.013010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:58.019828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:58.020764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:58.021714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:58.024352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 371/755] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:58.025439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.053876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:58.125476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 385/756] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:42:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:42:58.126273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.141566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d589277-5d02-4bcb-9139-4108eae9a501 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.188249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.189409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.189694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.190128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.203864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.205498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:58.221574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:58.223678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:58.225065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:58.232289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 372/757] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:58.232289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.250606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b629f8d-ef30-477b-b8b6-72e5ed825e7c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.269973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:58.289820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.290594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.291057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.291468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.292587 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:58.292901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.294399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.313245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 386/758] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:58.314369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.341759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:58.350940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fd65-a7fc-4d7a-bbec-20ccbe7dd576 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.389987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.391288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.391794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.392279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.404781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.406783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.423787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a762468-865f-4f8e-9b84-3e73fdea170c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:42:58.428927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:58.429886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:58.430812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:42:58.430812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:42:58.430812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:42:58.430812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:42:58.437568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 373/759] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:58.438753 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.463545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.464237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.464699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.465204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.466354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:58.466723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.468287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.489247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 387/760] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:58.489247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.529558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:58.573995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a7ea1-16c2-4db6-8281-ddb223d40e3c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.608946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee315-9bf0-4a0f-88d5-bb846030f57a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.610495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.611513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.611904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.612284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/f579f2db041f455786a02dcf621619ed/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.614995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=f579f2db041f455786a02dcf621619ed, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.616566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.619160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:58.637501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 374/761] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:42:58 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/f579f2db041f455786a02dcf621619ed/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:42:58.638833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.655975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.656573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.656953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.657278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.658064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:58.658473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.659882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.674648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 388/762] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:58.686619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:58.777507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cdd9b-3353-4228-8238-79bec4331029 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.821795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.822485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.822775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.823054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.823940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=0cbe226d7639456ebb94de533de30d28) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:58.824259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.825962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:58.832961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 375/763] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:58 2022] GET /identity/v3/role_assignments?user.id=0cbe226d7639456ebb94de533de30d28&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:58.897342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:58.906738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696a84-51a9-4811-ac84-4da826c98593 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:58.966797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88717556-cf81-4a8e-9a9c-bbb1fe914026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:58.968212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88717556-cf81-4a8e-9a9c-bbb1fe914026 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:58.968770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88717556-cf81-4a8e-9a9c-bbb1fe914026 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:58.969300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88717556-cf81-4a8e-9a9c-bbb1fe914026 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:58.982703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88717556-cf81-4a8e-9a9c-bbb1fe914026 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:58.984589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88717556-cf81-4a8e-9a9c-bbb1fe914026 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-88717556-cf81-4a8e-9a9c-bbb1fe914026 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:42:59.007702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:59.009209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:42:59.010127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:59.012817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 389/764] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:42:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:59.021185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:59.052751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b3a7-a713-4a16-ad48-bc8fa563b919 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.053483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b3a7-a713-4a16-ad48-bc8fa563b919 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.053973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b3a7-a713-4a16-ad48-bc8fa563b919 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.055817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 390/765] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:42:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:59.061559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268a35-de73-4bb0-86be-454d878f3479 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.062173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268a35-de73-4bb0-86be-454d878f3479 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.062680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268a35-de73-4bb0-86be-454d878f3479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.083451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-85268a35-de73-4bb0-86be-454d878f3479 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:59.084658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-85268a35-de73-4bb0-86be-454d878f3479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:59.098252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:59.162770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1143bf9-6c8b-4195-86c3-0a4071ab25b9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:59.186879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 391/766] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:42:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:42:59.219304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:59.219972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.220256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.220530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.221372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:59.221684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:59.223692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:59.237332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268a35-de73-4bb0-86be-454d878f3479 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:59.241698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 376/767] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:59.249184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:59.370808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98f964-a855-46c8-bfe4-1d860e196b33 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:59.389528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:59.408785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:59.409558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.409841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.410178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.411328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=3bf30288daec48dd8e04b96492e87341) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:59.411653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:59.413325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:59.420145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 377/768] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:42:59 2022] GET /identity/v3/role_assignments?user.id=3bf30288daec48dd8e04b96492e87341&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:59.450664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:59.479785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268a35-de73-4bb0-86be-454d878f3479 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:59.550390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:59.551779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.552061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.552331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.553005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:59.554453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:42:59.572535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:42:59.573795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:42:59.575262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 392/769] 10.222.0.91 () {56 vars in 1129 bytes} [Thu May 26 13:42:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:42:59.583262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:59.717810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d52b-7a2d-4168-9b76-ee74e8669ee0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:59.755241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:59.755917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.756248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.756523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.757291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:59.757598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:59.758975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:59.764638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 378/770] 10.222.0.91 () {56 vars in 1124 bytes} [Thu May 26 13:42:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:59.770212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:59.901930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98d366-24fb-49f7-b279-6f3216fb024c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:59.953105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:59.953811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.954092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.954368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.955084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:42:59.955387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:59.956772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:59.967597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 393/771] 10.222.0.91 () {56 vars in 1124 bytes} [Thu May 26 13:42:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:59.974213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:00.065608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:00.132660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de2771-dd01-46f3-b94b-c528f04f6b35 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:00.178370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53442425-62ba-494d-85b5-91bc23892a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:00.179054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53442425-62ba-494d-85b5-91bc23892a70 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:00.179334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53442425-62ba-494d-85b5-91bc23892a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:00.179609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53442425-62ba-494d-85b5-91bc23892a70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:00.180426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53442425-62ba-494d-85b5-91bc23892a70 None admin] RBAC: Adding query filter params (interface=public, service_id=5b45bcb1ec534959b5aea3967964a83b, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:00.180737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53442425-62ba-494d-85b5-91bc23892a70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:00.182149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53442425-62ba-494d-85b5-91bc23892a70 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:00.197182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 379/772] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:42:59 2022] GET /identity/v3/endpoints?service_id=5b45bcb1ec534959b5aea3967964a83b&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:00.198521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:00.234576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:00.302048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a254e1-661f-433b-812f-aea503ec77f5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:00.302369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a254e1-661f-433b-812f-aea503ec77f5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:00.302601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a254e1-661f-433b-812f-aea503ec77f5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:00.304371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 380/773] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:00.309911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570b199-16f2-421f-a014-e942f4de52e9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:00.310201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570b199-16f2-421f-a014-e942f4de52e9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:00.310481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570b199-16f2-421f-a014-e942f4de52e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:00.330937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3570b199-16f2-421f-a014-e942f4de52e9 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:00.336843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3570b199-16f2-421f-a014-e942f4de52e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:00.382696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15e857-ef85-4423-a269-7e030d294113 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:00.417821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:00.418503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:00.418792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:00.419075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:00.419764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:00.421175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:00.427681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 394/774] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:00 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:00.469201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 381/775] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:00.508493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:00.617535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:00.640056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac195f-128c-4fa4-9253-3efb7e7e9637 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:00.676943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:00.677794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:00.678371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:00.678653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] PATH_INFO: `/v3/services/network` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:00.679322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:00.680712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:00.693625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:00.694717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:00.696056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 395/776] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:00 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:00.702303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570b199-16f2-421f-a014-e942f4de52e9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:00.769462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:00.831679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570b199-16f2-421f-a014-e942f4de52e9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:00.867918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:00.868763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:00.869044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:00.869346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:00.870056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:00.870358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:00.871786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:00.877976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 382/777] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:00.883550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:01.006856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89249c7e-9022-413e-a24f-0c7178bf388e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:01.051495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:01.052104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:01.052388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:01.052665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:01.053426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] RBAC: Adding query filter params (type=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:01.053737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:01.055135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:01.065773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 396/778] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:01.076122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:01.256074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06948de7-6cef-4aee-9d42-2dcad348e98c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:01.261144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:01.309468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581a689-afdb-4c5b-897c-2fc211a35397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:01.310048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581a689-afdb-4c5b-897c-2fc211a35397 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:01.310329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581a689-afdb-4c5b-897c-2fc211a35397 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:01.310601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581a689-afdb-4c5b-897c-2fc211a35397 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:01.311398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581a689-afdb-4c5b-897c-2fc211a35397 None admin] RBAC: Adding query filter params (interface=public, service_id=517c17d601af4715a76df2ebdabaaa62, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:01.311703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581a689-afdb-4c5b-897c-2fc211a35397 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:01.313065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581a689-afdb-4c5b-897c-2fc211a35397 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:01.318942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 383/779] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:43:01 2022] GET /identity/v3/endpoints?service_id=517c17d601af4715a76df2ebdabaaa62&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:01.324308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:01.339923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:01.478559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bfa2e-2ac1-4b2d-b228-d529f9ad0a07 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:01.513864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d575809-c075-4a13-9c17-da5375492938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:01.514543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575809-c075-4a13-9c17-da5375492938 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:01.514824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575809-c075-4a13-9c17-da5375492938 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:01.515098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575809-c075-4a13-9c17-da5375492938 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:01.515788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d575809-c075-4a13-9c17-da5375492938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:01.517182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d575809-c075-4a13-9c17-da5375492938 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:01.523305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 397/780] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:01 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:01.606271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00bf71-7741-4a75-b2a3-64710996083f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:01.606873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00bf71-7741-4a75-b2a3-64710996083f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:01.607332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00bf71-7741-4a75-b2a3-64710996083f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:01.609437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 384/781] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:01.615490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60794a8-20d7-4641-9d9d-d88270023e5a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:01.615785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60794a8-20d7-4641-9d9d-d88270023e5a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:01.616063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60794a8-20d7-4641-9d9d-d88270023e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:01.639012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e60794a8-20d7-4641-9d9d-d88270023e5a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:01.640246 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e60794a8-20d7-4641-9d9d-d88270023e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:01.676321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666e6b-0009-488c-bf3d-38d55c1e29b0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:01.676635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666e6b-0009-488c-bf3d-38d55c1e29b0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:01.676885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666e6b-0009-488c-bf3d-38d55c1e29b0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:01.678673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 385/782] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:01.684690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3108821-49a5-446e-9cc3-99480eb4b0f4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:01.684969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3108821-49a5-446e-9cc3-99480eb4b0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:01.685308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3108821-49a5-446e-9cc3-99480eb4b0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:01.719604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e3108821-49a5-446e-9cc3-99480eb4b0f4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:01.720837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e3108821-49a5-446e-9cc3-99480eb4b0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:01.755747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:01.764981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 398/783] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:01.789179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60794a8-20d7-4641-9d9d-d88270023e5a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:01.842371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 386/784] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:01.870387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3108821-49a5-446e-9cc3-99480eb4b0f4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:01.959423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60794a8-20d7-4641-9d9d-d88270023e5a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.010956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.017384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3108821-49a5-446e-9cc3-99480eb4b0f4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.020659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.022009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.022291 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.022570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.045715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:02.053993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:02.054978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:02.055850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:02.056517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 399/785] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:02.073371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.084722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.085703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.086093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.086471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.101602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:02.111113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:02.112365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:02.113429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:02.114915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 387/786] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:02.121799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.224348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.318578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc176a7-de9c-4d7f-8212-225818394c5a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.322068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d62277-85c3-4907-9cfe-dcb139e2e008 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.360673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca06ac-02fd-4e39-b9b8-8c8b3508a803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.361486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca06ac-02fd-4e39-b9b8-8c8b3508a803 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.361776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca06ac-02fd-4e39-b9b8-8c8b3508a803 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.362056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca06ac-02fd-4e39-b9b8-8c8b3508a803 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.362356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d9aed-99a6-4542-b3ac-e9d57bd5c6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.362823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca06ac-02fd-4e39-b9b8-8c8b3508a803 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:02.363032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d9aed-99a6-4542-b3ac-e9d57bd5c6bb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.363644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca06ac-02fd-4e39-b9b8-8c8b3508a803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:02.363841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d9aed-99a6-4542-b3ac-e9d57bd5c6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.364122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d9aed-99a6-4542-b3ac-e9d57bd5c6bb None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.364869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d9aed-99a6-4542-b3ac-e9d57bd5c6bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:02.365039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca06ac-02fd-4e39-b9b8-8c8b3508a803 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:02.365225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d9aed-99a6-4542-b3ac-e9d57bd5c6bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:02.366772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d9aed-99a6-4542-b3ac-e9d57bd5c6bb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:02.374685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 400/787] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:02.375956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a449dc3-d226-4159-b11e-caafdc1f7080 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.376257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a449dc3-d226-4159-b11e-caafdc1f7080 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.376539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a449dc3-d226-4159-b11e-caafdc1f7080 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.378108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 401/788] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:02.381770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a449dc3-d226-4159-b11e-caafdc1f7080 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.387156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 388/789] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:02.388524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09adc-ebff-4d31-915d-ea7d874682f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.388820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09adc-ebff-4d31-915d-ea7d874682f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.389103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09adc-ebff-4d31-915d-ea7d874682f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.410235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6fb09adc-ebff-4d31-915d-ea7d874682f0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:02.411718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6fb09adc-ebff-4d31-915d-ea7d874682f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:02.431474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.493430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 389/790] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:02.494278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09adc-ebff-4d31-915d-ea7d874682f0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.509026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a449dc3-d226-4159-b11e-caafdc1f7080 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.534697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.549353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.551230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.551781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.552347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.570180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:02.572124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:02.591720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:02.592707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:02.593700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:02.593700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:02.593700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:02.593700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:02.596251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 402/791] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:02.597489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.606661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09adc-ebff-4d31-915d-ea7d874682f0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.648351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.649415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.650024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.650233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.662193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:02.663677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:02.681859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:02.682926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:02.683875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:02.683875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:02.683875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:02.683875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:02.687836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 390/792] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:02.688754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.691752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.721363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaaf25-362c-483d-89ae-7b5249275e68 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.761284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.762506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.763105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.763593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.775818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:02.784669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:02.785705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:43:02.786568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:02.789582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 403/793] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:02.790785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.796485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d9723-ce5a-4b35-8949-4f41daa270a8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.830185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.833499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.835163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.835446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.835718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.836445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:02.836749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:02.838162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:02.856595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 391/794] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:02.857640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.909302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc3f25-168e-4013-9b9d-a43c5e152cee None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.924014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.952425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.953680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.954290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.954779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.955724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:02.956365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:02.958175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:02.965873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefe1ed-b915-4403-a0aa-ef3e7ebb3170 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.972380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 404/795] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:02.974532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.008353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.009027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.009375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.009649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.010504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:03.010813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.012440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.022343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 392/796] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:03.023024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.093232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:03.104219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb292b8-b7f4-4a37-93ce-5cd10419e1a4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.141412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba914e2a-6174-4c48-b658-12b365555586 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.145040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.146291 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.146789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.147294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.156681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.158831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.170366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:03.179596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:03.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:03.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:03.184766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 405/797] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:03.186154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.209003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.209889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.210231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.210509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.219714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.221378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:03.237619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:03.239175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 393/798] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:03.240748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.310107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668025d-bc69-4750-88f9-4a8c13b6d964 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.349604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.350905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.351413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.351891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.364623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.366620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.368530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7e4ca-915f-4680-80e8-adceb7247921 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:03.386732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:03.387806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:03.388825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:03.388825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:03.388825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:03.388825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:03.388825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 406/799] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:03.389775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.399682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:03.408453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f32ab-413a-41d8-b63b-bc91b43ccb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.409128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f32ab-413a-41d8-b63b-bc91b43ccb47 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.409435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f32ab-413a-41d8-b63b-bc91b43ccb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.409737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f32ab-413a-41d8-b63b-bc91b43ccb47 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.410596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f32ab-413a-41d8-b63b-bc91b43ccb47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:03.410859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f32ab-413a-41d8-b63b-bc91b43ccb47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.412294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f32ab-413a-41d8-b63b-bc91b43ccb47 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.432659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 394/800] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:03.435096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34367f3-f7d0-4ed3-ae33-79f0f6ba9815 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.435771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34367f3-f7d0-4ed3-ae33-79f0f6ba9815 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.436465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34367f3-f7d0-4ed3-ae33-79f0f6ba9815 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.439456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 395/801] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:03.442088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34367f3-f7d0-4ed3-ae33-79f0f6ba9815 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.495192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:03.504112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73a106-8137-4893-93b7-1206b5a6af09 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.539948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.540622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.540907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.541193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.542030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:03.542337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.543766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.562417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34367f3-f7d0-4ed3-ae33-79f0f6ba9815 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.562739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 407/802] 10.222.0.91 () {56 vars in 1135 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:03.563565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.604666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7b78d-9017-4911-b277-9f3485205d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.605648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b78d-9017-4911-b277-9f3485205d7c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.605964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b78d-9017-4911-b277-9f3485205d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.606242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b78d-9017-4911-b277-9f3485205d7c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.606977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b78d-9017-4911-b277-9f3485205d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:03.607288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b78d-9017-4911-b277-9f3485205d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.608689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b78d-9017-4911-b277-9f3485205d7c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.626231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 396/803] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:03.626231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad12849-c4d9-4b62-b0cd-967523fba6a3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.626231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad12849-c4d9-4b62-b0cd-967523fba6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.626231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad12849-c4d9-4b62-b0cd-967523fba6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.641712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aad12849-c4d9-4b62-b0cd-967523fba6a3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:03.642917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aad12849-c4d9-4b62-b0cd-967523fba6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:03.643432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:03.697465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd01f0-4f35-481b-8875-458628e99d42 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.720768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 397/804] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:03.721983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad12849-c4d9-4b62-b0cd-967523fba6a3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.736803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.737853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.739007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.739510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.769957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:03.817822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.819395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.846174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad12849-c4d9-4b62-b0cd-967523fba6a3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:03.883950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:03.884751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:03.885033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:03.885346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:03.885571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:03.897312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:03.898823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:03.920275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:03.921477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:03.922679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:03.922679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:03.922679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:03.922679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:03.922679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 398/805] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:03.923001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:03.934797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 408/806] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:03.935826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.024424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.042109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8c915-f319-4465-982c-761404961bd4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.055502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbad38-325c-4506-aeab-ecaa5d710321 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.079386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.080210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.080493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.080765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.088040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.089512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.094096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.094918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.095204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.095522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.096146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.097588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:04.105250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:04.106244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:04.107214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:04.109458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 399/807] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:04.110391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:04.111347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:43:04.112193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:04.113752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.116869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 409/808] 10.222.0.91 () {56 vars in 1129 bytes} [Thu May 26 13:43:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:04.117783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.152103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.224361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309ef8b-4ab2-4f4b-9501-c8fec336415f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.245882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0a57d-e48b-43f9-9c5b-cc8cf8abf1bd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.260269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.260938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.261257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.261537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.262264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:04.262627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.263943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.275688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 400/809] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:04.276439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.281802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.288596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.289439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.289727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.290001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.305468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.306982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:04.325986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:04.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:04.327866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:04.327866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:04.327866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:04.327866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:04.327866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 410/810] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:04.328154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.392604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d72a13-a24c-46e9-aa30-6735f835129d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.423404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.436461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f812f0-0842-47e8-a842-60044f3b8ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.438396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f812f0-0842-47e8-a842-60044f3b8ebd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.439307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f812f0-0842-47e8-a842-60044f3b8ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.440065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f812f0-0842-47e8-a842-60044f3b8ebd None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.440932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f812f0-0842-47e8-a842-60044f3b8ebd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:04.441370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f812f0-0842-47e8-a842-60044f3b8ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.442747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f812f0-0842-47e8-a842-60044f3b8ebd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.449647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e21cf-0ff7-471b-b92e-c517aef1adec None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.461096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 401/811] 10.222.0.91 () {56 vars in 1149 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:04.462419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadeb0d-906c-4fd0-aff3-8d08ca72c8cf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.462707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadeb0d-906c-4fd0-aff3-8d08ca72c8cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.463123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadeb0d-906c-4fd0-aff3-8d08ca72c8cf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.464224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 402/812] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:04.465941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadeb0d-906c-4fd0-aff3-8d08ca72c8cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.503634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.504059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.504696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.504696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.505308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:04.505611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.507080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.511512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 411/813] 10.222.0.91 () {56 vars in 1124 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:04.512275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.532807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.576314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadeb0d-906c-4fd0-aff3-8d08ca72c8cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.628456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.629087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956db6-cdf4-4b9f-bf77-f94aa7914f99 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.629334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.629619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.629962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.639267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.640722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:04.656393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:04.658201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 403/814] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:04.660900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.670548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f34566-2d29-4346-8f8e-79077bf1340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.671212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f34566-2d29-4346-8f8e-79077bf1340a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.671494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f34566-2d29-4346-8f8e-79077bf1340a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.671766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f34566-2d29-4346-8f8e-79077bf1340a None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.672740 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f34566-2d29-4346-8f8e-79077bf1340a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:04.673042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f34566-2d29-4346-8f8e-79077bf1340a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.674448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f34566-2d29-4346-8f8e-79077bf1340a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.686879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 412/815] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:04.688101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a17015-2170-4ae7-b1dd-7007a9d310a0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.688385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a17015-2170-4ae7-b1dd-7007a9d310a0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.688661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a17015-2170-4ae7-b1dd-7007a9d310a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.695760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.707660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-73a17015-2170-4ae7-b1dd-7007a9d310a0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:04.708847 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-73a17015-2170-4ae7-b1dd-7007a9d310a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:04.776561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bfc89-2d89-43f7-8f36-d560735b7c45 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.788920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 413/816] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:04.789885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a17015-2170-4ae7-b1dd-7007a9d310a0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.805479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.817914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.818715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.818993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.819264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.829897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.831347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:04.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:04.850553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:04.851455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:04.851455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:04.851455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:04.851455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:04.851455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 404/817] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:04.851455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.898738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a17015-2170-4ae7-b1dd-7007a9d310a0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.942003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.942675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.942962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.943239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.943949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:04.944254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.945669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:04.946959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:04.957310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 414/818] 10.222.0.91 () {56 vars in 1124 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:04.958236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:04.960335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441029b-94c0-46a3-9ce3-d8edb9985a10 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:04.995646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:04.996345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:04.996627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:04.996935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:04.997792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:04.998105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:04.999546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.017258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 405/819] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:05.017258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.049817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:05.063493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41fe0b-f58d-4f26-b158-3c48c2b32d75 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.098450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.099247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.099533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.099814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.106722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.108172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.121824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af545edb-2950-495d-8b2b-12dce2f15388 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:05.122173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:05.123433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 415/820] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:05.124497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.163587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.164384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.164668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.164954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.165632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:05.179452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:05.180570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:05.181581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 406/821] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:05.182387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.203900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:05.260220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5eed56-bd14-49c8-853c-952325c35e1a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.285288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:05.290141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c8b13-9ca9-4a06-8f0e-2337efc42f5e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.296329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.297281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.297569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.297848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.298576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:05.298878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.300329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.312142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 416/822] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:05.312942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.327286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.328029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.328306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.328577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.329338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.330696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.351762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 407/823] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:05.352578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.435984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886e65a-34f1-4d6a-8296-ded17a4976ae None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.459474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:05.478942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11b635-2376-4149-9e1d-89314da0e7db None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.479494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.480415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.480700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.480978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/0cbe226d7639456ebb94de533de30d28/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.483122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=0cbe226d7639456ebb94de533de30d28, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.484636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.500168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 417/824] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:43:05 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/0cbe226d7639456ebb94de533de30d28/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:43:05.501292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.540285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.541233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.541587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.541960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.542851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:05.543174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.544674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.558837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 408/825] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:05.559675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.575525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:05.645118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752e31-0228-40be-a060-7c75b4231169 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.684924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.685657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.685945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.686226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.687033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] RBAC: Adding query filter params (name=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:05.687346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.688819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.689842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bcef1-c113-493c-bc1b-946a977d0347 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.694995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 418/826] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:05.695879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.749335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.750758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.751720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:05.751844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.752625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.764894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.767213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:05.789864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:05.791053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:05.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:05.797271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 409/827] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:05.798340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.836119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b3dd5-00a8-4620-b411-e78fa74b55fb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.875006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.875390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:05.876646 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.877088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.877641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.878830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=f579f2db041f455786a02dcf621619ed) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:05.879705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.881376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.897715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 419/828] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/role_assignments?user.id=f579f2db041f455786a02dcf621619ed&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:05.898946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:05.916354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9176c7e-f82b-41c7-967c-57920db4e720 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:05.959121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:05.960209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:05.960604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:05.960988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:05.961906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:05.962590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:05.964402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:05.978892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 410/829] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:05.984623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:06.037060 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63588c3a-fc25-4cc9-bd2c-6597d33279ce None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:06.046662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:06.083079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:06.084743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:06.085385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:06.086062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:06.087551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:06.088135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:06.090105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:06.109316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 420/830] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:06.112135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:06.165588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f59e0a-b061-4774-9905-1f624939d3f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:06.212972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfd86b-534d-49b2-bb52-191153932832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:06.214776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfd86b-534d-49b2-bb52-191153932832 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:06.215218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfd86b-534d-49b2-bb52-191153932832 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:06.215644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfd86b-534d-49b2-bb52-191153932832 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:06.216636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfd86b-534d-49b2-bb52-191153932832 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:06.218310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfd86b-534d-49b2-bb52-191153932832 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:06.223718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:06.246896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 411/831] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:43:06.261073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc9331-e441-4db7-bb2a-fbe04bf1f896 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:06.312507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca071a-6dbd-4351-a585-e117cac3d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:06.313601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca071a-6dbd-4351-a585-e117cac3d2e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:06.313883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca071a-6dbd-4351-a585-e117cac3d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:06.314157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca071a-6dbd-4351-a585-e117cac3d2e2 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/3bf30288daec48dd8e04b96492e87341/roles/f423841743584b218cefb221f7b3edff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:06.337368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca071a-6dbd-4351-a585-e117cac3d2e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=3bf30288daec48dd8e04b96492e87341, role_id=f423841743584b218cefb221f7b3edff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:06.339156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca071a-6dbd-4351-a585-e117cac3d2e2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:06.361388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 421/832] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:43:06 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/3bf30288daec48dd8e04b96492e87341/roles/f423841743584b218cefb221f7b3edff => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:43:07.821011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5266c10-3ea0-4c3f-aebd-831d7791fd43 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:07.821596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5266c10-3ea0-4c3f-aebd-831d7791fd43 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:07.821596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5266c10-3ea0-4c3f-aebd-831d7791fd43 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:07.823228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 412/833] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:07.829597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a74b5-eb60-4d0b-a180-5d81a4ca1561 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:07.830087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a74b5-eb60-4d0b-a180-5d81a4ca1561 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:07.830176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a74b5-eb60-4d0b-a180-5d81a4ca1561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:07.848280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a74a74b5-eb60-4d0b-a180-5d81a4ca1561 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:07.849509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a74a74b5-eb60-4d0b-a180-5d81a4ca1561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:07.857188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:07.884589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556cbe6-de8b-4932-98ff-2602a4a19834 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:07.884922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556cbe6-de8b-4932-98ff-2602a4a19834 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:07.885166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556cbe6-de8b-4932-98ff-2602a4a19834 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:07.886783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 413/834] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:07.896239 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813a060-2d30-4472-9cd6-c37c0c2c9a3f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:07.896546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813a060-2d30-4472-9cd6-c37c0c2c9a3f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:07.896796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813a060-2d30-4472-9cd6-c37c0c2c9a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:07.914841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1813a060-2d30-4472-9cd6-c37c0c2c9a3f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:07.916044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1813a060-2d30-4472-9cd6-c37c0c2c9a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:07.937770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 422/835] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:07.961629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a74b5-eb60-4d0b-a180-5d81a4ca1561 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:07.997844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 414/836] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:08.020685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813a060-2d30-4472-9cd6-c37c0c2c9a3f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.075807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a74b5-eb60-4d0b-a180-5d81a4ca1561 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.112818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.113663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.114103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.114383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.115053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:08.116457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:08.129073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:08.130204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:08.131122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:08.135167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 423/837] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:08.136435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.137506 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813a060-2d30-4472-9cd6-c37c0c2c9a3f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.147075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:08.163141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:08.181894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.183181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.183691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.184226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.196410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:08.206180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:08.207786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:08.208657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:08.216095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 415/838] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:08.217480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.245285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e60ae-af58-49f2-bdcc-86881eee5042 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.281791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.282477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.282741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.283024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.283723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:08.284029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:08.285459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:08.295156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 424/839] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:08.296465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.333710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24c950-db9b-491f-a728-63d5f31d7e1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.371640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:08.374447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.375529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.376031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.376521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.377771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:08.378348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:08.381429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:08.396789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 416/840] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:08.398180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.408168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6bb40-ab76-4f30-bbbe-daf93e494c1b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.445353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36302f-6130-4051-a7f2-89acec012a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.446031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36302f-6130-4051-a7f2-89acec012a06 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.446313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36302f-6130-4051-a7f2-89acec012a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.446633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36302f-6130-4051-a7f2-89acec012a06 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.447416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36302f-6130-4051-a7f2-89acec012a06 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:08.447708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36302f-6130-4051-a7f2-89acec012a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:08.449091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36302f-6130-4051-a7f2-89acec012a06 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:08.451825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:08.461333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 425/841] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:08.524055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a9b7-72bd-4cb5-852d-1ee4f99f6e9b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.567680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01893ec-485a-47af-8994-260462381e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.569006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01893ec-485a-47af-8994-260462381e3b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.569345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01893ec-485a-47af-8994-260462381e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.569602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01893ec-485a-47af-8994-260462381e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.570484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaba3c7-fc85-45b3-a9d8-71206800ddff None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.570779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaba3c7-fc85-45b3-a9d8-71206800ddff None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.571060 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaba3c7-fc85-45b3-a9d8-71206800ddff None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.572783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 426/842] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:08.578586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f353b3-e807-4321-bbee-56921304d04c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.579213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f353b3-e807-4321-bbee-56921304d04c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.579716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f353b3-e807-4321-bbee-56921304d04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.585602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01893ec-485a-47af-8994-260462381e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:08.587983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01893ec-485a-47af-8994-260462381e3b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:08.598699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c5f353b3-e807-4321-bbee-56921304d04c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:08.600289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c5f353b3-e807-4321-bbee-56921304d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b01893ec-485a-47af-8994-260462381e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:08.608996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:08.610111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:08.611034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:08.611034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:08.611034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:08.611034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:08.611034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 417/843] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:08.617420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01893ec-485a-47af-8994-260462381e3b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.699386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 427/844] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:08.719303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d1d6-3fd2-4151-8700-4b271234f718 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.719592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d1d6-3fd2-4151-8700-4b271234f718 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.719873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d1d6-3fd2-4151-8700-4b271234f718 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.721700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 428/845] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:08.726182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d1d6-3fd2-4151-8700-4b271234f718 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.739264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:08.747279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01893ec-485a-47af-8994-260462381e3b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.786900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ae841-ff5b-40d7-b4b8-0358f7ae9523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.787568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ae841-ff5b-40d7-b4b8-0358f7ae9523 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.787846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ae841-ff5b-40d7-b4b8-0358f7ae9523 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.788117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ae841-ff5b-40d7-b4b8-0358f7ae9523 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.788841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ae841-ff5b-40d7-b4b8-0358f7ae9523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:08.789138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ae841-ff5b-40d7-b4b8-0358f7ae9523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:08.790533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ae841-ff5b-40d7-b4b8-0358f7ae9523 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:08.801632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:08.803451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 418/846] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:08.804815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eac35-26e6-48ad-8f0b-15200f1be554 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.805095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eac35-26e6-48ad-8f0b-15200f1be554 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.805555 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eac35-26e6-48ad-8f0b-15200f1be554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.821816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0f8eac35-26e6-48ad-8f0b-15200f1be554 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:08.822983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f8eac35-26e6-48ad-8f0b-15200f1be554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:08.847720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d1d6-3fd2-4151-8700-4b271234f718 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:08.885227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:08.886047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:08.886329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:08.886601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:08.887260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:08.888832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:08.901280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 419/847] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:08.901280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eac35-26e6-48ad-8f0b-15200f1be554 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:08.901623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:08.902522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:08.903383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:08.907677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 429/848] 10.222.0.91 () {56 vars in 1132 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:08.908908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:08.996939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.014844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eac35-26e6-48ad-8f0b-15200f1be554 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.016201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30381c3a-0570-41e2-b274-32cfaf450523 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.044778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.053434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.054240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.054520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.054793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.055367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.056035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.056318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.056585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.057322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:09.057635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.059005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.063473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.068379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 430/849] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:09.068379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.068620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:09.083264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:09.084463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:09.085480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 420/850] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:09.086123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.184474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ed6c4-ca22-4e8c-a2fb-f997b6806209 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.203592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b583-5df7-497c-a391-2fe344ddb84c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.224831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.226059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.226690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.227237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.238773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:09.240521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.241209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.241494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.241767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.242468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:09.242771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.244127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:09.246910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:09.248220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 431/851] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:09.249081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.254741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 421/852] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:09.255444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.272174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.327946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.390476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba802b-9f9f-405a-a357-3f9280afc71c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.398944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29fab7-a739-431d-abee-a684e83b657f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.431455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.432204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.432533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.432812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.433615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:09.433918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.435352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.442029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.443033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.443320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.443597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.444438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:09.444747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.445002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 422/853] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:09.446220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.451011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.466791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 432/854] 10.222.0.91 () {56 vars in 1135 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:09.475088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.530090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.572491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e295c-becd-455d-b68f-2435d07619c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.597626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725cd31-d0b7-4249-9300-9dca6ab81b27 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.612658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.614102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.614859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.615110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.615789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.639802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.642763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.645255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.646180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.647056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.647541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:09.677520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:09.678527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:09.679443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:09.679443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:09.679443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:09.679443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:09.685577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 423/855] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:09.689489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.730472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.732590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.839160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b076f-6533-47d3-b8dc-bc2476a5e62a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:09.872472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 433/856] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:09.875069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:09.879648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.888325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:09.898847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:09.899948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:09.900482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:09.900961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:09.902585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:09.903104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:09.905126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:09.923498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 424/857] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:09.930289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.017420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bc3cc-f9a7-4707-8db1-d0a8df3be3fe None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.041486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe21820-3f8b-4a47-82ce-54aff801a565 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.068367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.070110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.070869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.071338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.078029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.078924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.079270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.079666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.085616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.087630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:10.089595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.090050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:10.113372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:10.114448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:10.115511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:10.118913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:10.120527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:10.121465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:10.121465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:10.121465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:10.121465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:10.121724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 425/858] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:10.121724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.128245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 434/859] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:10.130100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.184269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:10.220764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:10.260972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21d1dc-19bd-485b-bfc2-35e67bbd9c59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.280032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e9ac2-f5ee-4a7e-8bb5-fd257d34b52a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.314416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.315501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.316686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.317200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.318111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.318519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:10.318785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.319066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.319343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.319678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.320065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:10.320367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.321805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:10.321924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:10.340554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 435/860] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:10.341661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.350564 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 426/861] 10.222.0.91 () {56 vars in 1149 bytes} [Thu May 26 13:43:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:10.357511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.490318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88a600-7b8e-4aac-b144-f40fb6498381 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.530173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:10.549748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:10.555859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd5b26-e9fd-4fea-ad26-5f626d2b50d2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.557777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.560005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.561917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.562987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.579770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.584907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:10.618435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:10.620027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:10.620934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:10.628700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 436/862] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:10.630432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.639101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9ad1e-479b-4026-9636-8f2976784a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.639768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9ad1e-479b-4026-9636-8f2976784a4b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.640051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9ad1e-479b-4026-9636-8f2976784a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.640325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9ad1e-479b-4026-9636-8f2976784a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.719478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9ad1e-479b-4026-9636-8f2976784a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.720932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9ad1e-479b-4026-9636-8f2976784a4b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:10.762773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7be0a-6daf-4216-badb-ba44f6296112 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.808283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2eee2-d616-4481-9d39-9b03f2df8c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.809441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2eee2-d616-4481-9d39-9b03f2df8c06 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.809751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2eee2-d616-4481-9d39-9b03f2df8c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.810008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2eee2-d616-4481-9d39-9b03f2df8c06 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.810792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2eee2-d616-4481-9d39-9b03f2df8c06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:10.811105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2eee2-d616-4481-9d39-9b03f2df8c06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.812557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2eee2-d616-4481-9d39-9b03f2df8c06 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:10.829931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 437/863] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:10.829931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c33df-b6af-4842-a3c0-b4fa6519637c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.829931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c33df-b6af-4842-a3c0-b4fa6519637c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.829931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c33df-b6af-4842-a3c0-b4fa6519637c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.829931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 438/864] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:10.834393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c33df-b6af-4842-a3c0-b4fa6519637c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.856032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:10.865576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 427/865] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:10.866922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff02d-6dfb-479c-be29-b20783ef75b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.867853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff02d-6dfb-479c-be29-b20783ef75b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.867963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff02d-6dfb-479c-be29-b20783ef75b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.884413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-361ff02d-6dfb-479c-be29-b20783ef75b3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:10.885610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-361ff02d-6dfb-479c-be29-b20783ef75b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:10.889600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:10.950779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c33df-b6af-4842-a3c0-b4fa6519637c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:10.953110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 428/866] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:10.958054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ff02d-6dfb-479c-be29-b20783ef75b3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:10.994605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:10.995266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:10.995547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:10.996440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:10.996650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=0cbe226d7639456ebb94de533de30d28) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:10.997285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:10.998432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.009187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 439/867] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:10 2022] GET /identity/v3/role_assignments?user.id=0cbe226d7639456ebb94de533de30d28&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:11.009187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:11.086236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ff02d-6dfb-479c-be29-b20783ef75b3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:11.120299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:11.127211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:11.128184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.128648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.129023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.141655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:11.143260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.157280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cd0d5-cb84-4c34-a3c6-0daa7455c43f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:11.162475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:11.163576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:11.164686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:11.164686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:11.164686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:11.164686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:11.167578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:11.169652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 429/868] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:11.171493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:11.206543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:11.207413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.207844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.208216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.209081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:11.211605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.233931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 440/869] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:43:11 2022] POST /identity/v3/services => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:11.319007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaed1b-61c9-4d23-be00-cc63945ff366 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:11.368922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a92948-96e1-44d9-9722-59650494e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:11.370049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a92948-96e1-44d9-9722-59650494e9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.370543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a92948-96e1-44d9-9722-59650494e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.371027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a92948-96e1-44d9-9722-59650494e9c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.372003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a92948-96e1-44d9-9722-59650494e9c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:11.372580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a92948-96e1-44d9-9722-59650494e9c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:11.374281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a92948-96e1-44d9-9722-59650494e9c2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.389897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 430/870] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:11.399747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:11.503756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c4a7-28dc-4266-ad47-8b43042e43f8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:11.539772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:11.540579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.540861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.541141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.548655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:11.550152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.550935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bf93e-1947-4b94-b97e-bd24d28d5f17 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.551224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bf93e-1947-4b94-b97e-bd24d28d5f17 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.551501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bf93e-1947-4b94-b97e-bd24d28d5f17 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.553087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 431/871] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:11.558124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471e43-e733-4279-99f0-2802ad0f8a5a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.558405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471e43-e733-4279-99f0-2802ad0f8a5a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.558676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471e43-e733-4279-99f0-2802ad0f8a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:11.564669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:11.565647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:11.566586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 441/872] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:11.571694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:11.578714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-06471e43-e733-4279-99f0-2802ad0f8a5a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:11.580200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-06471e43-e733-4279-99f0-2802ad0f8a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:11.601460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:11.618551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:11.669493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 432/873] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:11.677494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad09d91-b8e5-4cdb-88bf-599bf641ca91 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:11.684580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471e43-e733-4279-99f0-2802ad0f8a5a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:11.714820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:11.715612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.715899 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.716181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.717265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:11.717581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:11.719309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.735581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 442/874] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:11.741331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:11.795110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471e43-e733-4279-99f0-2802ad0f8a5a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:11.833719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:11.834278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bbf5f-d218-4040-aad9-f0746e54b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:11.835279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bbf5f-d218-4040-aad9-f0746e54b0ed None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.835768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bbf5f-d218-4040-aad9-f0746e54b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.836247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bbf5f-d218-4040-aad9-f0746e54b0ed None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.837292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bbf5f-d218-4040-aad9-f0746e54b0ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:11.838873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bbf5f-d218-4040-aad9-f0746e54b0ed None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.845569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f28b92-b306-40b3-a363-48dc173962d3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:11.865940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 433/875] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:43:11 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:43:11.891209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6bf24-db95-4be8-a3cf-f44c18403475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:11.891938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6bf24-db95-4be8-a3cf-f44c18403475 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:11.892150 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6bf24-db95-4be8-a3cf-f44c18403475 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:11.892434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6bf24-db95-4be8-a3cf-f44c18403475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:11.893314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6bf24-db95-4be8-a3cf-f44c18403475 None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=3bf30288daec48dd8e04b96492e87341) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:11.893629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6bf24-db95-4be8-a3cf-f44c18403475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:11.895694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6bf24-db95-4be8-a3cf-f44c18403475 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:11.903089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 443/876] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:11 2022] GET /identity/v3/role_assignments?user.id=3bf30288daec48dd8e04b96492e87341&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:13.390291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2502-f84e-498e-9ec0-705f7e71fdc4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:13.390844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2502-f84e-498e-9ec0-705f7e71fdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:13.390844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2502-f84e-498e-9ec0-705f7e71fdc4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:13.392446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 434/877] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:13.398546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6b0e3-0aee-40d4-8653-535dca390ad5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:13.398946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6b0e3-0aee-40d4-8653-535dca390ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:13.399134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6b0e3-0aee-40d4-8653-535dca390ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:13.415816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-53b6b0e3-0aee-40d4-8653-535dca390ad5 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:13.416987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-53b6b0e3-0aee-40d4-8653-535dca390ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:13.489304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 444/878] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:13.506848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2502-f84e-498e-9ec0-705f7e71fdc4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:13.537799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:13.567423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3f662-c843-4df3-832f-0bd706355284 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:13.567733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3f662-c843-4df3-832f-0bd706355284 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:13.568029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3f662-c843-4df3-832f-0bd706355284 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:13.569697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 445/879] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:13.576112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4a15-1d03-4778-b4ee-6570169a7a5e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:13.576704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4a15-1d03-4778-b4ee-6570169a7a5e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:13.577057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4a15-1d03-4778-b4ee-6570169a7a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:13.594698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9b6b4a15-1d03-4778-b4ee-6570169a7a5e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:13.596002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9b6b4a15-1d03-4778-b4ee-6570169a7a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:13.626186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2502-f84e-498e-9ec0-705f7e71fdc4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:13.664322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:13.665137 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:13.665447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:13.665723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:13.666383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:13.667763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:13.673422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 446/880] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:13.681789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:13.682823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:13.684216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 435/881] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:13.690523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4a15-1d03-4778-b4ee-6570169a7a5e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:13.697071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:13.702756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:13.802853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4a15-1d03-4778-b4ee-6570169a7a5e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:13.823906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:13.835298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e079a5-d049-45b3-9a05-f2612d97278c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:13.845274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:13.846467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:13.847009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:13.847499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:13.848434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:13.850036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:13.864773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:13.866097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:13.866949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:13.870039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 447/882] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:13.875722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:13.879890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:13.880622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:13.880959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:13.881274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:13.882041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:13.882342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:13.883708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:13.893285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 436/883] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:13.894794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:13.954603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:13.990644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e2886-821d-4bc1-b4a7-ef45120f6136 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.008031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07326ba6-846f-467a-9f19-8db0349889e4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.033032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.034028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.034990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.035559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.036574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:14.037075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:14.038704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:14.049701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 448/884] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:14.051558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.053824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c3508-9a21-4365-a356-4c5ac77bdc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.056692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c3508-9a21-4365-a356-4c5ac77bdc1c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.056692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c3508-9a21-4365-a356-4c5ac77bdc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.056692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c3508-9a21-4365-a356-4c5ac77bdc1c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.057109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c3508-9a21-4365-a356-4c5ac77bdc1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:14.057464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c3508-9a21-4365-a356-4c5ac77bdc1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:14.058947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c3508-9a21-4365-a356-4c5ac77bdc1c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:14.071721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 437/885] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:14.169830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de480-82fe-4327-a848-7bd6d4c2e810 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.207936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cbcb0-263c-426d-bead-3bc560f42929 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.208285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cbcb0-263c-426d-bead-3bc560f42929 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.208502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cbcb0-263c-426d-bead-3bc560f42929 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.210209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 438/886] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:14.215821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0227ee-720c-4926-b755-43238569a5e1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.216108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0227ee-720c-4926-b755-43238569a5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.216384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0227ee-720c-4926-b755-43238569a5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.224846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f1999-d536-4748-bcbf-4fa086a791f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.225424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1999-d536-4748-bcbf-4fa086a791f7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.225712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1999-d536-4748-bcbf-4fa086a791f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.225992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1999-d536-4748-bcbf-4fa086a791f7 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.226720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f1999-d536-4748-bcbf-4fa086a791f7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:14.227030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f1999-d536-4748-bcbf-4fa086a791f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:14.228504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f1999-d536-4748-bcbf-4fa086a791f7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:14.230044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:14.235177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 449/887] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:14.236579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc0df0-5ad8-4d60-842f-57fcdf8bf9ae None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.236875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc0df0-5ad8-4d60-842f-57fcdf8bf9ae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.237185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc0df0-5ad8-4d60-842f-57fcdf8bf9ae None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.239406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 450/888] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:14.241064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc0df0-5ad8-4d60-842f-57fcdf8bf9ae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.246588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6a0227ee-720c-4926-b755-43238569a5e1 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:14.247824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a0227ee-720c-4926-b755-43238569a5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:14.264998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:14.330761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 439/889] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:14.332975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105920d-9759-4f2d-bf0e-4f41ce18c605 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.333344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105920d-9759-4f2d-bf0e-4f41ce18c605 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.333720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105920d-9759-4f2d-bf0e-4f41ce18c605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.356715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4105920d-9759-4f2d-bf0e-4f41ce18c605 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:14.357532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc0df0-5ad8-4d60-842f-57fcdf8bf9ae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.357946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4105920d-9759-4f2d-bf0e-4f41ce18c605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:14.399999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.400697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.400950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.401253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.402055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] RBAC: Adding query filter params (interface=public, service_id=57f487c493f24287931543a9879cd6f6, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:14.402348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:14.404334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:14.412890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 451/890] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/endpoints?service_id=57f487c493f24287931543a9879cd6f6&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:14.414069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.449199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 440/891] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:14.449199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105920d-9759-4f2d-bf0e-4f41ce18c605 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.491715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:14.530324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:14.546063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b852-ca82-4149-b058-7f4dcf738414 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.554093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105920d-9759-4f2d-bf0e-4f41ce18c605 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.580198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.581075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.581846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.582139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.582800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:14.584174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:14.592027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.592704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.592984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.593354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.593983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:14.595372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:14.596261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:14.597823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:14.598695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 452/892] 10.222.0.91 () {56 vars in 1132 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:14.599553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.613189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 441/893] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:43:14.613189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.725083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156770a2-812e-44f5-b7c8-df4b16aa811e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.755448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ad147-abe2-486f-bd12-7a17d88b4663 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.759696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:14.770995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.771830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.772599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.772892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.788002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:14.799698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:14.801724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:14.803444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 453/894] 10.222.0.91 () {56 vars in 1120 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:14.809376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.811057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.811994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.812532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.813047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.813995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:14.814635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:14.816872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:14.824952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 442/895] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:14.830993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:14.852704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:14.949195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47447e9-97c9-40bf-a6cc-aa46fb815c67 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.979718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc088c1-34ba-4ab1-a4b7-8a950567d32e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:14.996986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:14.997941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:14.998376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:14.998799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:14.999719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:15.000174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.001764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.011728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 454/896] 10.222.0.91 () {56 vars in 1115 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:15.020237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:15.020383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.020383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.020593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.020858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:15.021602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:15.021912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.023385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.033409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 443/897] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:15.034804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:15.072851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:15.113427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:15.146267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3e235-f857-4585-8d7c-608092b7277c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:15.152181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99492bed-6367-4076-9f8f-fd29c2d708b7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:15.199455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.200420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.200818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.201242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:15.209067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.210061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.210499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.210963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:15.211946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] RBAC: Adding query filter params (interface=public, service_id=cdd423b4b7b44293bca7b5ea38d5975e, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:15.212553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.214999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.220218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.223424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 444/898] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:43:15 2022] GET /identity/v3/endpoints?service_id=cdd423b4b7b44293bca7b5ea38d5975e&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:15.223687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.230959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:15.257814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:15.258824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:15.259724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:15.259724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:15.259724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:15.259724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:15.263377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 455/899] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:15.264944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:15.362461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169ffcd-4036-438b-9913-bef0b4384691 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:15.374491 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1184e-0d0a-4357-80fb-3c1f1c8f7afd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:15.397331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:15.402894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.403660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.403999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.404273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:15.405313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.407927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.410485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:15.418119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 445/900] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:15 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:43:15.420580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.422200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.422734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.423221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:15.424347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:15.424884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.427120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.444990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 456/901] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:15.450789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:15.570504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33ba1d-2636-4d21-b2cb-53d689dd8825 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:15.608380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.609237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.609507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.609777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:15.617330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.618776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:15.634713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:15.635682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:15.636652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 446/902] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:43:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:15.641966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:15.747081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10773874-d6c6-41a8-ae88-d68268151e68 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:15.783368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.784041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.784324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.784634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:15.785505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:15.785816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:15.787245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:15.806171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 457/903] 10.222.0.91 () {56 vars in 1149 bytes} [Thu May 26 13:43:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:15.811758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:15.876878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:15.923600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96f3d5-4de3-43f2-afdc-5f08cc519a4c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:15.964737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:15.965441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:15.965728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:15.966012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.040072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:16.041573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:16.102606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:16.146251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 447/904] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:15 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:16.151739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:16.174297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:16.265294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf50b-03af-487e-b17b-33025016e6cd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:16.302394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7176e-a1fd-4af6-82a1-cc87afb94f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:16.303208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7176e-a1fd-4af6-82a1-cc87afb94f03 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.303492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7176e-a1fd-4af6-82a1-cc87afb94f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.303770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7176e-a1fd-4af6-82a1-cc87afb94f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.315145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7176e-a1fd-4af6-82a1-cc87afb94f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:16.316655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7176e-a1fd-4af6-82a1-cc87afb94f03 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-05f7176e-a1fd-4af6-82a1-cc87afb94f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:16.335026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:16.336011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:16.336966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:16.336966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:16.336966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:16.336966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:16.336966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 458/905] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:16.342236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:16.358140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c9a75-b176-4e9c-a5b0-2aee79d5c5e7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.358983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c9a75-b176-4e9c-a5b0-2aee79d5c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.359444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c9a75-b176-4e9c-a5b0-2aee79d5c5e7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.361928 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 459/906] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:16.367802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992b6e3-cf1e-4693-bf8c-58167ef9c724 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.368091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992b6e3-cf1e-4693-bf8c-58167ef9c724 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.368368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992b6e3-cf1e-4693-bf8c-58167ef9c724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.390396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b992b6e3-cf1e-4693-bf8c-58167ef9c724 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:16.391594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b992b6e3-cf1e-4693-bf8c-58167ef9c724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:16.467058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:16.467714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688dc95-6a79-40fe-9ab3-ed74ed7c7c8f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:16.481233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 460/907] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:16.497524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992b6e3-cf1e-4693-bf8c-58167ef9c724 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:16.509193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:16.509868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.510153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.510429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.511607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:16.511919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:16.513783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:16.530014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 448/908] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:16.537100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:16.562821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:16.614499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992b6e3-cf1e-4693-bf8c-58167ef9c724 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:16.649512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbab49a-969b-43e4-977e-104a3dc751b5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:16.652284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:16.653217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.653706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.654199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.655151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:16.656910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:16.683858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 461/909] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:43:16 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:16.703240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:16.704116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.704405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.704726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.713646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:16.715199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:16.732339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:16.733349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:16.734686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 449/910] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:16.739887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:16.817235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:16.869380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062699c-0926-4fbe-9ce2-3172213f563c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:16.906571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb13e4-9584-46de-910e-a331a2b7952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:16.907247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb13e4-9584-46de-910e-a331a2b7952f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.907533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb13e4-9584-46de-910e-a331a2b7952f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.907813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb13e4-9584-46de-910e-a331a2b7952f None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.908670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb13e4-9584-46de-910e-a331a2b7952f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:16.908983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb13e4-9584-46de-910e-a331a2b7952f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:16.911086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb13e4-9584-46de-910e-a331a2b7952f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:16.924470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 462/911] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:16.930414 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:16.997079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8c59-68d8-48f9-9a44-c87fb1298177 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:16.997324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8c59-68d8-48f9-9a44-c87fb1298177 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:16.997607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8c59-68d8-48f9-9a44-c87fb1298177 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:16.998994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 463/912] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:17.004674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de49be-a426-4738-b140-97871e275127 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.004968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de49be-a426-4738-b140-97871e275127 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.005299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de49be-a426-4738-b140-97871e275127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.025604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-58de49be-a426-4738-b140-97871e275127 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:17.027135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-58de49be-a426-4738-b140-97871e275127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:17.045538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:17.055716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4372d-0fff-45b5-8dbd-21eafc04fcc3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:17.100587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:17.101275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.101558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.101858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.102701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] RBAC: Adding query filter params (role.id=5687582c149649509b22380e30cd1947, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=3bf30288daec48dd8e04b96492e87341) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:17.102997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:17.103476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 464/913] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:17.104470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:17.118204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 450/914] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:16 2022] GET /identity/v3/role_assignments?user.id=3bf30288daec48dd8e04b96492e87341&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=5687582c149649509b22380e30cd1947 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:17.132185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de49be-a426-4738-b140-97871e275127 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:17.193939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:17.251012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de49be-a426-4738-b140-97871e275127 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:17.290849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:17.291678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.291961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.292238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.292907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:17.294595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:17.308329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:17.309423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:17.310689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 465/915] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:17.318238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:17.433341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff42f74-f2f7-49a9-814a-d4ad56f1882e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:17.470590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:17.472733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.473271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.473811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.474720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:17.475229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:17.476796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:17.487117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 451/916] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:17.488749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:17.600184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd0c09-3c14-40f8-a901-2f096e06558d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:17.615070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:17.638645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8521a96-a7d8-45a0-b965-13be03470758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:17.639333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521a96-a7d8-45a0-b965-13be03470758 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.639691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521a96-a7d8-45a0-b965-13be03470758 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.639972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521a96-a7d8-45a0-b965-13be03470758 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.640757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8521a96-a7d8-45a0-b965-13be03470758 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:17.641072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8521a96-a7d8-45a0-b965-13be03470758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:17.642524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8521a96-a7d8-45a0-b965-13be03470758 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:17.649209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 466/917] 10.222.0.91 () {56 vars in 1114 bytes} [Thu May 26 13:43:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:17.654902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:17.659242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:17.762250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90fc16-b05a-4df2-b1bb-f781ea479776 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:17.797960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9227c-84d8-47f6-984f-e09249e7a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:17.798628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9227c-84d8-47f6-984f-e09249e7a60d None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.798914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9227c-84d8-47f6-984f-e09249e7a60d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.799192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9227c-84d8-47f6-984f-e09249e7a60d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.799980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9227c-84d8-47f6-984f-e09249e7a60d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:17.801433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9227c-84d8-47f6-984f-e09249e7a60d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:17.826485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 452/918] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:17.840455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff4f88-511e-447b-affb-7c2642ac3212 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.840811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff4f88-511e-447b-affb-7c2642ac3212 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.841027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff4f88-511e-447b-affb-7c2642ac3212 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.843596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 467/919] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:17.850685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a147f74-11d6-433f-9398-6f96eb53e80f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:17.851045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a147f74-11d6-433f-9398-6f96eb53e80f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:17.851467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a147f74-11d6-433f-9398-6f96eb53e80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:17.876256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7a147f74-11d6-433f-9398-6f96eb53e80f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:17.877782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7a147f74-11d6-433f-9398-6f96eb53e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:17.956329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:17.976747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 453/920] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:18.000477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff4f88-511e-447b-affb-7c2642ac3212 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:18.111724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff4f88-511e-447b-affb-7c2642ac3212 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:18.147522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:18.148393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:18.148685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:18.149002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:18.149687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:18.151060 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:18.166029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:18.167014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:43:18.167869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:18.169224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 468/921] 10.222.0.91 () {56 vars in 1132 bytes} [Thu May 26 13:43:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:18.176561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a147f74-11d6-433f-9398-6f96eb53e80f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:18.289779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a147f74-11d6-433f-9398-6f96eb53e80f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:18.326131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:18.326808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:18.327093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:18.327369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:18.328084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:18.328394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:18.329801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:18.335215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 454/922] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:18.340497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:18.389613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:18.450367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51818e08-754a-48b5-91fd-92c1414aa12c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:18.485127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de354d-8161-4e0c-a43f-26aa0b536275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:18.486255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de354d-8161-4e0c-a43f-26aa0b536275 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:18.486790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de354d-8161-4e0c-a43f-26aa0b536275 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:18.487273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de354d-8161-4e0c-a43f-26aa0b536275 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:18.488831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de354d-8161-4e0c-a43f-26aa0b536275 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:18.489800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de354d-8161-4e0c-a43f-26aa0b536275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:18.491396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de354d-8161-4e0c-a43f-26aa0b536275 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:18.497144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 469/923] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:18.502372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:18.576528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:18.613185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dd528-41ee-4516-b74e-19088a046e3f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:18.648347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78310479-9a69-4a23-9581-44bd353de04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:18.648939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310479-9a69-4a23-9581-44bd353de04a None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:18.649250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310479-9a69-4a23-9581-44bd353de04a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:18.649532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310479-9a69-4a23-9581-44bd353de04a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:18.650279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78310479-9a69-4a23-9581-44bd353de04a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:18.651657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78310479-9a69-4a23-9581-44bd353de04a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:18.673815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 455/924] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:19.088958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3cfee-ef44-4142-b35c-6a1a74e73be7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.089509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3cfee-ef44-4142-b35c-6a1a74e73be7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.089908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3cfee-ef44-4142-b35c-6a1a74e73be7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.091707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 470/925] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:19.097833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb97bd5-58e2-4d65-891a-3af97d162883 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.098312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb97bd5-58e2-4d65-891a-3af97d162883 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.098696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb97bd5-58e2-4d65-891a-3af97d162883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.115968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7eb97bd5-58e2-4d65-891a-3af97d162883 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:19.117177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7eb97bd5-58e2-4d65-891a-3af97d162883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:19.197476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 456/926] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:19.223106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3cfee-ef44-4142-b35c-6a1a74e73be7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:19.349306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3cfee-ef44-4142-b35c-6a1a74e73be7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:19.357300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:19.390347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b22d2f-df1d-49ed-9dc1-9e886d03628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:19.391174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b22d2f-df1d-49ed-9dc1-9e886d03628a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.391457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b22d2f-df1d-49ed-9dc1-9e886d03628a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.391730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b22d2f-df1d-49ed-9dc1-9e886d03628a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.392390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b22d2f-df1d-49ed-9dc1-9e886d03628a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:19.394364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b22d2f-df1d-49ed-9dc1-9e886d03628a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-24b22d2f-df1d-49ed-9dc1-9e886d03628a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:19.407365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:19.408410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:19.409344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 471/927] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:19.414872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb97bd5-58e2-4d65-891a-3af97d162883 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:19.462670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:19.531320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb97bd5-58e2-4d65-891a-3af97d162883 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:19.558826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a41c5-6c6f-4c38-8f72-be4eed869c32 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.559474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a41c5-6c6f-4c38-8f72-be4eed869c32 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.559474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a41c5-6c6f-4c38-8f72-be4eed869c32 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.561005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 472/928] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:19.566829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb84fcd-320b-42d4-94f2-abf4bacd5338 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.567111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb84fcd-320b-42d4-94f2-abf4bacd5338 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.567385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb84fcd-320b-42d4-94f2-abf4bacd5338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.575014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:19.575700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.575986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.576262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.576995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:19.577375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:19.579216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:19.584730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5bb84fcd-320b-42d4-94f2-abf4bacd5338 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:19.585628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 457/929] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:19.585936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5bb84fcd-320b-42d4-94f2-abf4bacd5338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:19.591161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:19.662396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 473/930] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:19.686833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb84fcd-320b-42d4-94f2-abf4bacd5338 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:19.689871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:19.697969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b706c44-c09b-48b6-b192-7ee9567ff6d6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:19.733294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:19.734068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.734426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.734773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.735558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:19.735940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:19.737932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:19.743443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 458/931] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:19.749729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:19.794590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb84fcd-320b-42d4-94f2-abf4bacd5338 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:19.797463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:19.833457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:19.834271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.834549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.834820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.846745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:19.855674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:19.857850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:19.859844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:19.869444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 474/932] 10.222.0.91 () {56 vars in 1120 bytes} [Thu May 26 13:43:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:19.869444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:19.869750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe70e69-12a4-431f-ab30-48f7fbaf7947 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:19.903102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:19.903797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:19.904132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:19.904413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:19.905250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] RBAC: Adding query filter params (interface=public, service_id=ffd3249c055a47ea992566039e690e07, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:19.905567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:19.907076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:19.917180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 459/933] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:43:19 2022] GET /identity/v3/endpoints?service_id=ffd3249c055a47ea992566039e690e07&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:19.918005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:19.946693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:20.009469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee03e6-ca68-4146-be3f-d7501821d883 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.036815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fca18-cf5e-4d48-bd3d-e416e305b7c3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.057094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.057874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.058229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.058574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.060102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:20.060760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.062903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.073737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680a956-5bba-45c1-aae9-496387ca9424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.074410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680a956-5bba-45c1-aae9-496387ca9424 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.074697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680a956-5bba-45c1-aae9-496387ca9424 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.074978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680a956-5bba-45c1-aae9-496387ca9424 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.075674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680a956-5bba-45c1-aae9-496387ca9424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.077060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680a956-5bba-45c1-aae9-496387ca9424 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.078875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 475/934] 10.222.0.91 () {56 vars in 1115 bytes} [Thu May 26 13:43:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:20.080230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.089475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 460/935] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:19 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:43:20.109443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:20.188857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c81d3e-2e2a-4141-9278-85101903880c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.194356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c81d3e-2e2a-4141-9278-85101903880c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.194356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c81d3e-2e2a-4141-9278-85101903880c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.195350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 461/936] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:20.201175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fc0d-4995-47e4-a1fb-acaa4e0254d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.201175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fc0d-4995-47e4-a1fb-acaa4e0254d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.201175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fc0d-4995-47e4-a1fb-acaa4e0254d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.201678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14361966-36bb-4bbd-afca-095abc246ce4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.216069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a185fc0d-4995-47e4-a1fb-acaa4e0254d5 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:20.217587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a185fc0d-4995-47e4-a1fb-acaa4e0254d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:20.245669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.246642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.247075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.247463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.260483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.262168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:20.283608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:20.287290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:20.290360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:20.290360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:20.290360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:20.290360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:20.290360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 476/937] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:20.294677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.339744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 462/938] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:20.365997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185fc0d-4995-47e4-a1fb-acaa4e0254d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.385020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:20.389110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:20.425195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f47f-bd37-4760-af7a-167b80f02eb9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.476145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.477644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.478218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.478764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.479636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:20.480087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.480594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185fc0d-4995-47e4-a1fb-acaa4e0254d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.482294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.498708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 477/939] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:20.505320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.528110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.529060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.529528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.529947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.530792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.532371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.534894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING keystone.server.flask.common [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 13:43:20.555048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 13:43:20.555048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 13:43:20.555048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [parameters: {'id': 'fc8a8d180daa4344ac622ade65781086', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 13:43:20.555048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fc8a8d180daa4344ac622ade65781086.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fc8a8d180daa4344ac622ade65781086. May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:43:20.563975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 13:43:20.564903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() May 26 13:43:20.565949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 13:43:20.566854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 13:43:20.567862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:43:20.568671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': 'fc8a8d180daa4344ac622ade65781086', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:20.570509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:20.571889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fc8a8d180daa4344ac622ade65781086. May 26 13:43:20.573500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.576288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 463/940] 10.222.0.91 () {60 vars in 1140 bytes} [Thu May 26 13:43:20 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:20.584683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.632327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a108e7a-2678-4b38-955d-2d2fca8ce2c3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.673800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.674709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.675087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.675374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.677543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:20.684562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.686887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.701804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:20.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:20.703749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:20.704720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:20.709258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 478/941] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:43:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:20.710197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.740729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8c715-88a5-43d8-83d3-0f5843c125f4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.783884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.785000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.785620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.786041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.795449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.797874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:20.817527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:20.818618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:20.819705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:20.820445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b981d-ad82-4449-bb66-985e27992b9c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.821979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 464/942] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:20.828183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.871920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:20.872630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:20.872912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:20.873230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:20.874111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:20.874423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:20.875915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:20.905276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 479/943] 10.222.0.91 () {56 vars in 1149 bytes} [Thu May 26 13:43:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:20.906354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:20.958478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f1da7-22e9-4acb-8a1b-d9a3f26d5f88 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:20.968003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:21.002662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e1a4-75e5-4bfa-b9e7-0cb9b1aceac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.004291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e1a4-75e5-4bfa-b9e7-0cb9b1aceac0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.004780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e1a4-75e5-4bfa-b9e7-0cb9b1aceac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.005304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e1a4-75e5-4bfa-b9e7-0cb9b1aceac0 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.006432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e1a4-75e5-4bfa-b9e7-0cb9b1aceac0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:21.006955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e1a4-75e5-4bfa-b9e7-0cb9b1aceac0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.008707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e1a4-75e5-4bfa-b9e7-0cb9b1aceac0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.010651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:21.030392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c3676-2489-4063-b3d1-c5c73bfd8a8a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.037551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 465/944] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:21.061101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd48f-8e6d-49ba-9866-a909ce68fc21 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.061474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd48f-8e6d-49ba-9866-a909ce68fc21 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.061757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd48f-8e6d-49ba-9866-a909ce68fc21 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.066369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 466/945] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:21.069983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.070804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.070913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b1423-5295-48d7-a89f-edcdfbf64194 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.071144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.071248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b1423-5295-48d7-a89f-edcdfbf64194 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.071422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.071520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b1423-5295-48d7-a89f-edcdfbf64194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.085562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.087168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.089609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-872b1423-5295-48d7-a89f-edcdfbf64194 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:21.091167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-872b1423-5295-48d7-a89f-edcdfbf64194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:21.112896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:21.113921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:21.114860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:21.114860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:21.114860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:21.114860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:21.114860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 480/946] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:21.121134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:21.185547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 467/947] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:21.205689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b1423-5295-48d7-a89f-edcdfbf64194 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:21.266749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4678ffb-9653-4112-875a-54f2dc218a71 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.270540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:21.310684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.311530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.311817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.312096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.312828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:21.313138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.314564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.329683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 481/948] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:21.338453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:21.352550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b1423-5295-48d7-a89f-edcdfbf64194 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.378842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:21.391366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.392358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.392713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.393049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.394326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.395759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:21.408319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:21.410156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:21.411218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:21.412781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 468/949] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:21.418861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:21.473789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e4645-ed36-412e-a7c4-cb292997d85a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.519660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.521344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.521921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.522476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.530534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7aded-c086-48af-bdbc-fb5f0ff78e2b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.531572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.534433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:21.552729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:21.553736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:21.554824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:21.562181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 482/950] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:21.562988 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:21.578757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.580106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.580590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.581845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.583160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:21.583627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.585586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.592125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 469/951] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:21.597716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:21.598558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:21.664260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:21.695460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202ee3a-ea53-44c4-88c3-d2f280ac6b80 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.724547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f0a54-f4c2-4bf1-a3d6-9191afb47e6e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.737992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.738662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.738944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.739218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.740007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:21.740313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.741803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.758336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 483/952] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:21.760075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:21.764336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b67857-487d-40a9-afc3-6913378bb0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.765436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b67857-487d-40a9-afc3-6913378bb0df None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.765436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b67857-487d-40a9-afc3-6913378bb0df None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.765650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b67857-487d-40a9-afc3-6913378bb0df None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.766292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b67857-487d-40a9-afc3-6913378bb0df None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:21.766599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b67857-487d-40a9-afc3-6913378bb0df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.767974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b67857-487d-40a9-afc3-6913378bb0df None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.781413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 470/953] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:21.859937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:21.878941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b0978-e3b3-4e9f-9be1-fc2dd51ff65a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:21.916554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd121-f8a3-44c4-aaad-7e49cc845e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:21.917879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd121-f8a3-44c4-aaad-7e49cc845e52 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:21.918368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd121-f8a3-44c4-aaad-7e49cc845e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:21.918888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd121-f8a3-44c4-aaad-7e49cc845e52 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/3bf30288daec48dd8e04b96492e87341/roles/5687582c149649509b22380e30cd1947` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:21.922177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd121-f8a3-44c4-aaad-7e49cc845e52 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=3bf30288daec48dd8e04b96492e87341, role_id=5687582c149649509b22380e30cd1947)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:21.924011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd121-f8a3-44c4-aaad-7e49cc845e52 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:21.943312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 484/954] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:43:21 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/3bf30288daec48dd8e04b96492e87341/roles/5687582c149649509b22380e30cd1947 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:43:22.421925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e1cc6-c106-417a-96c7-4ebf331e652a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:22.422290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e1cc6-c106-417a-96c7-4ebf331e652a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:22.422497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e1cc6-c106-417a-96c7-4ebf331e652a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:22.424155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 471/955] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:22.430237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7fb00-8200-4541-90be-2bf70ecdd825 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:22.430536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7fb00-8200-4541-90be-2bf70ecdd825 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:22.430824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7fb00-8200-4541-90be-2bf70ecdd825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:22.447962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e9d7fb00-8200-4541-90be-2bf70ecdd825 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:22.449090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e9d7fb00-8200-4541-90be-2bf70ecdd825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:22.520623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 485/956] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:22.543473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e1cc6-c106-417a-96c7-4ebf331e652a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:22.656259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e1cc6-c106-417a-96c7-4ebf331e652a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:22.675828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:22.697489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:22.698446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:22.698672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:22.698946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:22.699604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:22.700972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:22.714258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:22.715405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:22.716306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 472/957] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:22.721539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7fb00-8200-4541-90be-2bf70ecdd825 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:22.799216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:22.825143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7fb00-8200-4541-90be-2bf70ecdd825 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:22.865606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:22.866529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:22.866976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:22.867393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:22.868854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:22.869355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:22.871999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:22.877654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 486/958] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:22.883233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:22.990911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3021ca8-e097-4f69-9235-df7ae84f510d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:23.026317 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d6e30-fcc1-4422-af51-0b1f1dd56e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:23.027116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d6e30-fcc1-4422-af51-0b1f1dd56e34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:23.027407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d6e30-fcc1-4422-af51-0b1f1dd56e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:23.027684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d6e30-fcc1-4422-af51-0b1f1dd56e34 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:23.028434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d6e30-fcc1-4422-af51-0b1f1dd56e34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:23.028744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d6e30-fcc1-4422-af51-0b1f1dd56e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:23.030208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d6e30-fcc1-4422-af51-0b1f1dd56e34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:23.036357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 473/959] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:23.042197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:23.146612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038ab37-8337-4fe2-8516-89dbf551015f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:23.197930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466b3fd-61da-4865-a4be-e38b18dcb985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:23.198934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466b3fd-61da-4865-a4be-e38b18dcb985 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:23.199427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466b3fd-61da-4865-a4be-e38b18dcb985 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:23.199902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466b3fd-61da-4865-a4be-e38b18dcb985 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:23.200903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466b3fd-61da-4865-a4be-e38b18dcb985 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:23.202568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466b3fd-61da-4865-a4be-e38b18dcb985 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:23.212214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:23.231127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 487/960] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:23.408807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6737598-b9c6-45a9-8466-ae149135d0e4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:23.409131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6737598-b9c6-45a9-8466-ae149135d0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:23.409358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6737598-b9c6-45a9-8466-ae149135d0e4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:23.410885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 474/961] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:23.416450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c415b-9ef6-4215-b69d-60d66da2341e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:23.416747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c415b-9ef6-4215-b69d-60d66da2341e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:23.417025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c415b-9ef6-4215-b69d-60d66da2341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:23.435160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e45c415b-9ef6-4215-b69d-60d66da2341e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:23.436325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e45c415b-9ef6-4215-b69d-60d66da2341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:23.512966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 488/962] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:23.537124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6737598-b9c6-45a9-8466-ae149135d0e4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:23.591949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:23.652060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6737598-b9c6-45a9-8466-ae149135d0e4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:23.688290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:23.689054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:23.689282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:23.689563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:23.690295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:23.691808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:23.710571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 13:43:23.710571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 13:43:23.710571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [parameters: {'user_id': 'aa3ddfd663ae4750b8f127b2a4ccd671', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 13:43:23.710571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 26 13:43:23.718846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:43:23.719760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 13:43:23.720849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 13:43:23.722652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:43:23.723842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:43:23.724898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'user_id': 'aa3ddfd663ae4750b8f127b2a4ccd671', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:23.726019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 13:43:23.727153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 13:43:23.728320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 13:43:23.728320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 13:43:23.728320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 13:43:23.728320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 13:43:23.728320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 13:43:23.728320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:23.728320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 475/963] 10.222.0.91 () {60 vars in 1131 bytes} [Thu May 26 13:43:23 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:23.737406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c415b-9ef6-4215-b69d-60d66da2341e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:23.845663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c415b-9ef6-4215-b69d-60d66da2341e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:23.867279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:23.883744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a27475-09b2-4bf0-9053-f48aaa5dbf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:23.884554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a27475-09b2-4bf0-9053-f48aaa5dbf3c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:23.884844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a27475-09b2-4bf0-9053-f48aaa5dbf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:23.885132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a27475-09b2-4bf0-9053-f48aaa5dbf3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:23.893392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a27475-09b2-4bf0-9053-f48aaa5dbf3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:23.895549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a27475-09b2-4bf0-9053-f48aaa5dbf3c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f1a27475-09b2-4bf0-9053-f48aaa5dbf3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:23.911019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:23.912042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:23.913074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 489/964] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:43:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:23.920252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:24.026618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e632c-2ea5-4f48-ad53-59aa1d3abb82 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:24.037790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:24.070393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e5ef5-5f0d-4c58-bac0-940c3fca88f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:24.071102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e5ef5-5f0d-4c58-bac0-940c3fca88f5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.071398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e5ef5-5f0d-4c58-bac0-940c3fca88f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.071755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e5ef5-5f0d-4c58-bac0-940c3fca88f5 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.072961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e5ef5-5f0d-4c58-bac0-940c3fca88f5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:24.074066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e5ef5-5f0d-4c58-bac0-940c3fca88f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:24.077256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e5ef5-5f0d-4c58-bac0-940c3fca88f5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:24.122352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 476/965] 10.222.0.91 () {56 vars in 1101 bytes} [Thu May 26 13:43:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:24.266499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab296ae-33c7-4b1d-82b7-7feb4ad4a5ec None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.266842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab296ae-33c7-4b1d-82b7-7feb4ad4a5ec None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.267102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab296ae-33c7-4b1d-82b7-7feb4ad4a5ec None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.269353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 490/966] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:24.275400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7240e6-cdff-4016-9ab4-e4441f1c4ebe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.275685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7240e6-cdff-4016-9ab4-e4441f1c4ebe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.275968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7240e6-cdff-4016-9ab4-e4441f1c4ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.290697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd34e-f9d5-4826-9a41-bbe0d0c8481d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.290888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd34e-f9d5-4826-9a41-bbe0d0c8481d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.291334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd34e-f9d5-4826-9a41-bbe0d0c8481d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.293185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 491/967] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:24.297067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e7240e6-cdff-4016-9ab4-e4441f1c4ebe None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:24.298793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e7240e6-cdff-4016-9ab4-e4441f1c4ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:24.300495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c94e0-c159-491d-b252-9c38886e2fcf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.300797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c94e0-c159-491d-b252-9c38886e2fcf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.301078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c94e0-c159-491d-b252-9c38886e2fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.320314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b82c94e0-c159-491d-b252-9c38886e2fcf None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:24.321606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b82c94e0-c159-491d-b252-9c38886e2fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:24.385613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 477/968] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:24.407662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 492/969] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:24.416376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7240e6-cdff-4016-9ab4-e4441f1c4ebe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:24.433836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c94e0-c159-491d-b252-9c38886e2fcf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:24.519924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:24.527252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7240e6-cdff-4016-9ab4-e4441f1c4ebe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:24.554938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c94e0-c159-491d-b252-9c38886e2fcf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:24.564597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:24.565807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.566114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.566552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:24.566729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.567577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:24.569100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:24.583573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 478/970] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:43:24 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:24.602604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:24.603482 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.603779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.604052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.619710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:24.628724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:24.629801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:24.630656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:24.633509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 493/971] 10.222.0.91 () {56 vars in 1120 bytes} [Thu May 26 13:43:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:24.636177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:24.752043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca9049-a488-43e1-9386-23bb16e5b138 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:24.792877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:24.793856 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.794306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.794584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.795338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:24.795646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:24.797324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:24.806077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 479/972] 10.222.0.91 () {56 vars in 1115 bytes} [Thu May 26 13:43:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:24.811895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:24.915350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af4f34-b3cc-4256-9fa0-de653d332058 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:24.951555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:24.952355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:24.952631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:24.952897 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:24.963684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:24.965134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:24.982766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:24.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:24.984656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:24.984656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:24.984656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:24.984656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:24.984656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 494/973] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:24.991681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:25.014650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:25.103338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef6e1-e5a4-4c3e-a3c9-f7cd15bad9af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:25.138601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:25.139317 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:25.139602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:25.139879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:25.140597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:25.140904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:25.142296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:25.158885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 480/974] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:25.160212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:25.181139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:25.271770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f630601-5b24-4845-a3a5-e0122f137534 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:25.311006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:25.311826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:25.312107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:25.312381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:25.322484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:25.325091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:25.345400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:25.347544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:43:25.348999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:25.350863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 495/975] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:43:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:25.352941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:25.452131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:25.479501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c1d1e-899a-4215-857c-78b45131ab50 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:25.519848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6690ca0-e246-4280-833f-bcf2771a47f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:25.520053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6690ca0-e246-4280-833f-bcf2771a47f1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:25.520305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6690ca0-e246-4280-833f-bcf2771a47f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:25.520574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6690ca0-e246-4280-833f-bcf2771a47f1 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:25.524074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6690ca0-e246-4280-833f-bcf2771a47f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:25.524394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6690ca0-e246-4280-833f-bcf2771a47f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:25.525892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6690ca0-e246-4280-833f-bcf2771a47f1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:25.544631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 481/976] 10.222.0.91 () {56 vars in 1149 bytes} [Thu May 26 13:43:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:25.550368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:25.559668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c1f4-acc7-4c3a-ab2c-27160e9909ae None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:25.559961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c1f4-acc7-4c3a-ab2c-27160e9909ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:25.560229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c1f4-acc7-4c3a-ab2c-27160e9909ae None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:25.561761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 482/977] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:25.569173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3aed-a8cc-443b-bbf1-fff07b76df83 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:25.569173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3aed-a8cc-443b-bbf1-fff07b76df83 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:25.569173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3aed-a8cc-443b-bbf1-fff07b76df83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:25.606171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-804e3aed-a8cc-443b-bbf1-fff07b76df83 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:25.608116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-804e3aed-a8cc-443b-bbf1-fff07b76df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:25.671018 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:25.694641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 483/978] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:25.699187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606095c-787b-4f1a-9964-cca357835445 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:25.720661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e3aed-a8cc-443b-bbf1-fff07b76df83 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:25.738313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:25.739275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:25.739739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:25.740194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:25.794709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:25.822616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:25.824533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:25.836058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e3aed-a8cc-443b-bbf1-fff07b76df83 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:25.873793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:25.874611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:25.874895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:25.875171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:25.875832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:25.877235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:25.889257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:25.890289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:25.891173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 484/979] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:25.897271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:25.915374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:25.940566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 496/980] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:25 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:25.946200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.013032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd7f8d-e014-49a9-bb0f-d1b9ae45c53a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.047130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:26.052401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.053115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.053428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.053701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.054455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:26.054761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.056118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.059778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786704af-fbb0-4847-9514-ce19b16713d0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.061637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 485/981] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:26.067972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.103025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.103821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.104101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.104372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.115566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.117005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:26.135231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:26.136179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:26.137096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:26.137096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:26.137096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:26.137096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:26.140863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 497/982] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:26.141729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.173568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:26.177273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a131547-be84-426f-b6b1-1b16b1acea57 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.220733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.221439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.221786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.222071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.222780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:26.223088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.224461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.229561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 486/983] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:26.235487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.251598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce0d6e-0e43-4d55-aea2-82bf823daf5f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.292743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.293768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.294237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.294707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.295953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:26.296450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.299060 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.304606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:26.318341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 498/984] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:26.323752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.360656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70633d6e-5c1e-429e-adc9-c94b5588e52a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.395931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.396607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.396896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.397194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.398001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] RBAC: Adding query filter params (interface=admin, service_id=ffd3249c055a47ea992566039e690e07, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:26.398319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.399695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.409350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 487/985] 10.222.0.91 () {56 vars in 1249 bytes} [Thu May 26 13:43:26 2022] GET /identity/v3/endpoints?service_id=ffd3249c055a47ea992566039e690e07&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:26.410541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.427383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:26.438606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc195c-3c8b-4b35-a76a-1109c6fca466 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.476612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.477443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.477729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.478009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.485392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.486848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:26.502044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:26.502954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:26.503965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 499/986] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:26.508968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.520850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175e481-df51-4114-acce-4d7198358d16 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.541120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:26.557825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.558424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.558722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.558990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.559674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.561043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.567344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 488/987] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:26 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:43:26.629978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c8adf-2445-489d-b2d4-d460e73ed5b8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.668871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80956af3-d4ed-42f9-a5de-eff1d327b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.669605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80956af3-d4ed-42f9-a5de-eff1d327b8ca None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.669891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80956af3-d4ed-42f9-a5de-eff1d327b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.670168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80956af3-d4ed-42f9-a5de-eff1d327b8ca None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.670997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80956af3-d4ed-42f9-a5de-eff1d327b8ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:26.671309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80956af3-d4ed-42f9-a5de-eff1d327b8ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.673098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80956af3-d4ed-42f9-a5de-eff1d327b8ca None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.687139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 500/988] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:26.693993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:26.809226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa8647-36b9-47bd-a94a-709a58412461 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:26.845258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49015515-4e92-4de5-8461-ada077145f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:26.846223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015515-4e92-4de5-8461-ada077145f9f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.846693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015515-4e92-4de5-8461-ada077145f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.847179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015515-4e92-4de5-8461-ada077145f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.848192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49015515-4e92-4de5-8461-ada077145f9f None admin] RBAC: Adding query filter params (role.id=5687582c149649509b22380e30cd1947, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=3bf30288daec48dd8e04b96492e87341) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:26.848699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49015515-4e92-4de5-8461-ada077145f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:26.850346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49015515-4e92-4de5-8461-ada077145f9f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:26.856509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 489/989] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:26 2022] GET /identity/v3/role_assignments?user.id=3bf30288daec48dd8e04b96492e87341&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=5687582c149649509b22380e30cd1947 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:26.871595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d4bb-d194-4208-a1cb-8c4336ca5a3d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.872138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d4bb-d194-4208-a1cb-8c4336ca5a3d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.872138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d4bb-d194-4208-a1cb-8c4336ca5a3d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.873658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 501/990] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:26.880139 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac54ed-fef4-4801-bc07-b306acd13e2e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:26.880439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac54ed-fef4-4801-bc07-b306acd13e2e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:26.880702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac54ed-fef4-4801-bc07-b306acd13e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:26.899403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-60ac54ed-fef4-4801-bc07-b306acd13e2e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:26.900622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60ac54ed-fef4-4801-bc07-b306acd13e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:26.951974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:26.978885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 490/991] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:27.002194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d4bb-d194-4208-a1cb-8c4336ca5a3d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:27.016268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:27.117044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d4bb-d194-4208-a1cb-8c4336ca5a3d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:27.152203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:27.153188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:27.153475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:27.153753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:27.154403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:27.155777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:27.168430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:27.169789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:27.170664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 502/992] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:27.178078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac54ed-fef4-4801-bc07-b306acd13e2e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:27.282621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac54ed-fef4-4801-bc07-b306acd13e2e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:27.317774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:27.318608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:27.318992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:27.319375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:27.320184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:27.320616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:27.322133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:27.329328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 491/993] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:27.333798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:27.412996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:27.439832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cf27b-5291-45d5-abb3-147f2d41a377 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:27.473541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:27.474279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:27.474625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:27.474963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:27.475739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:27.476106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:27.477563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:27.487326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 503/994] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:27.488605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:27.537844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:27.596723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f7a02-43a8-4a4a-a964-eccf89867a18 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:27.632241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a412d-e2e7-4066-8800-276c21d5f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:27.632930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a412d-e2e7-4066-8800-276c21d5f07a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:27.633240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a412d-e2e7-4066-8800-276c21d5f07a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:27.633517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a412d-e2e7-4066-8800-276c21d5f07a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:27.634301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a412d-e2e7-4066-8800-276c21d5f07a None admin] RBAC: Adding query filter params (interface=public, service_id=30768e5654754abc8e732307152a06d6, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:27.634609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a412d-e2e7-4066-8800-276c21d5f07a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:27.635974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a412d-e2e7-4066-8800-276c21d5f07a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:27.645274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 492/995] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:43:27 2022] GET /identity/v3/endpoints?service_id=30768e5654754abc8e732307152a06d6&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:27.647792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:27.750289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff57e4-338e-4426-ad07-153ca9d86cd3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:27.784818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917fcfd-a45d-4f67-8052-88b54518fcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:27.785524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917fcfd-a45d-4f67-8052-88b54518fcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:27.785812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917fcfd-a45d-4f67-8052-88b54518fcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:27.786089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917fcfd-a45d-4f67-8052-88b54518fcc8 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:27.786765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917fcfd-a45d-4f67-8052-88b54518fcc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:27.788135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917fcfd-a45d-4f67-8052-88b54518fcc8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:27.801237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 504/996] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:27 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:43:28.829595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b958cb-c830-40b7-9283-5ced358a9147 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:28.830142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b958cb-c830-40b7-9283-5ced358a9147 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:28.830142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b958cb-c830-40b7-9283-5ced358a9147 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:28.832181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 493/997] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:28.837410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce9e93-44ea-4a74-aae3-addcf316e863 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:28.837852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce9e93-44ea-4a74-aae3-addcf316e863 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:28.837997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce9e93-44ea-4a74-aae3-addcf316e863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:28.843066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:28.856114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5cce9e93-44ea-4a74-aae3-addcf316e863 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:28.857318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5cce9e93-44ea-4a74-aae3-addcf316e863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:28.931390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 505/998] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:28.954371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b958cb-c830-40b7-9283-5ced358a9147 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:29.057647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b958cb-c830-40b7-9283-5ced358a9147 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:29.066795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:29.084641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed30e6-f345-485b-abdd-d39700e0e591 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.084943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed30e6-f345-485b-abdd-d39700e0e591 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.085332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed30e6-f345-485b-abdd-d39700e0e591 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.086840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 506/999] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:29.092245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa603ff-0eee-4861-b6e4-9b8252234aa2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.092597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa603ff-0eee-4861-b6e4-9b8252234aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.092876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa603ff-0eee-4861-b6e4-9b8252234aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.100053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:29.100868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.101168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.101448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.102105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:29.103879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:29.109047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2fa603ff-0eee-4861-b6e4-9b8252234aa2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:29.110258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2fa603ff-0eee-4861-b6e4-9b8252234aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:29.116039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:29.117064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:29.122742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 494/1000] 10.222.0.91 () {56 vars in 1126 bytes} [Thu May 26 13:43:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:29.123783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:29.198309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 507/1001] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:29.220371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa603ff-0eee-4861-b6e4-9b8252234aa2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:29.244148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9b68d-3012-4348-9867-77838e929dde None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:29.288255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:29.288663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:29.288922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.289590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.290309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.290601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:29.290909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:29.292277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:29.301941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 495/1002] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:29.303164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:29.339664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:29.342340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa603ff-0eee-4861-b6e4-9b8252234aa2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:29.382482 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:29.383283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.383570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.383845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.384933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:29.386813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:29.401851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:29.403966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:29.405578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 508/1003] 10.222.0.91 () {56 vars in 1105 bytes} [Thu May 26 13:43:29 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:29.409410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:29.415732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead71e-5ad0-4984-8404-3a5c7881c3da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:29.452609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:29.453370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.453598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.453875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.454584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:29.454899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:29.456268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:29.461788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 496/1004] 10.222.0.91 () {56 vars in 1122 bytes} [Thu May 26 13:43:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:29.467188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:29.520880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303e30c-6b9a-4db5-9e32-2bd38615dcc5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:29.550363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:29.560611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:29.561610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.562105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.562592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.563485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] RBAC: Adding query filter params (name=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:29.563977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:29.566192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:29.571460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 509/1005] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:29.577079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:29.578997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:29.583465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a382-40cc-4cdb-8b4c-680e6bc6d20d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:29.619534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821a28c-553f-4e54-adce-b2a02beb5542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:29.620119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821a28c-553f-4e54-adce-b2a02beb5542 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.620402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821a28c-553f-4e54-adce-b2a02beb5542 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.620672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821a28c-553f-4e54-adce-b2a02beb5542 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.621468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821a28c-553f-4e54-adce-b2a02beb5542 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:29.622839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821a28c-553f-4e54-adce-b2a02beb5542 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:29.649182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 497/1006] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:43:29.703076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1127c-89a6-4ded-94bb-bf67db889f2b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:29.748326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603c10c-d504-4c5e-bc52-03951a5114fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:29.749008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c10c-d504-4c5e-bc52-03951a5114fc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:29.749330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c10c-d504-4c5e-bc52-03951a5114fc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:29.749610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c10c-d504-4c5e-bc52-03951a5114fc None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:29.750323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603c10c-d504-4c5e-bc52-03951a5114fc None admin] RBAC: Adding query filter params (type=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:29.750635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603c10c-d504-4c5e-bc52-03951a5114fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:29.752032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603c10c-d504-4c5e-bc52-03951a5114fc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:29.757457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 510/1007] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:30.025743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74c322-e665-4749-95d6-64415f011a41 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:30.026296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74c322-e665-4749-95d6-64415f011a41 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:30.026296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74c322-e665-4749-95d6-64415f011a41 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:30.027838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 498/1008] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:30.032814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6827232-8701-4eda-a0d7-b820f9ab7f6c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:30.033237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6827232-8701-4eda-a0d7-b820f9ab7f6c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:30.033402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6827232-8701-4eda-a0d7-b820f9ab7f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:30.052154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d6827232-8701-4eda-a0d7-b820f9ab7f6c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:30.053408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d6827232-8701-4eda-a0d7-b820f9ab7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:30.101168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:30.122441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 511/1009] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:30.145812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74c322-e665-4749-95d6-64415f011a41 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:30.257000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74c322-e665-4749-95d6-64415f011a41 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:30.291826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:30.292542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:30.292828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:30.293105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:30.293786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:30.295161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:30.307802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:30.309002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:43:30.309921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:30.313501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 499/1010] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:30.318072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6827232-8701-4eda-a0d7-b820f9ab7f6c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:30.425549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6827232-8701-4eda-a0d7-b820f9ab7f6c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:30.462560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:30.463234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:30.463519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:30.463796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:30.464506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:30.464819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:30.466236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:30.476257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 512/1011] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:30.477826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:30.504182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:30.585665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107176d-aa78-483e-a862-0e78d60cc810 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:30.620320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360165e-7a78-4d7a-943e-d41726b45126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:30.620985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360165e-7a78-4d7a-943e-d41726b45126 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:30.621307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360165e-7a78-4d7a-943e-d41726b45126 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:30.621583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360165e-7a78-4d7a-943e-d41726b45126 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:30.622288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360165e-7a78-4d7a-943e-d41726b45126 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:30.622669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360165e-7a78-4d7a-943e-d41726b45126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:30.624051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360165e-7a78-4d7a-943e-d41726b45126 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:30.633548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 500/1012] 10.222.0.91 () {56 vars in 1112 bytes} [Thu May 26 13:43:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:30.635028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:30.683327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:30.741175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4d4e4-b24a-42a0-a626-637fe9a79c87 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:30.779354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c44a0-d19e-47cf-abc0-95acf096c9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:30.780055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c44a0-d19e-47cf-abc0-95acf096c9b6 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:30.780342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c44a0-d19e-47cf-abc0-95acf096c9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:30.780708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c44a0-d19e-47cf-abc0-95acf096c9b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:30.782173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c44a0-d19e-47cf-abc0-95acf096c9b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:30.784730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c44a0-d19e-47cf-abc0-95acf096c9b6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:30.813530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 513/1013] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:31.872144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dccb39-b2e8-424c-a899-c02f10482ef0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:31.872676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dccb39-b2e8-424c-a899-c02f10482ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:31.872676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dccb39-b2e8-424c-a899-c02f10482ef0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:31.874130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 501/1014] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:31.879858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d93c4c-91da-41b5-b1c7-6669bafca373 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:31.880279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d93c4c-91da-41b5-b1c7-6669bafca373 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:31.880480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d93c4c-91da-41b5-b1c7-6669bafca373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:31.897350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-25d93c4c-91da-41b5-b1c7-6669bafca373 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:31.898548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-25d93c4c-91da-41b5-b1c7-6669bafca373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:31.969820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae241650-a38f-4cd4-b27c-690ce67a94b0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:31.970151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae241650-a38f-4cd4-b27c-690ce67a94b0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:31.970393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae241650-a38f-4cd4-b27c-690ce67a94b0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:31.972429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 502/1015] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:31.977687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b709a-92d2-43f0-8fdd-8b012243033f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:31.978035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 514/1016] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:31 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:31.981586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b709a-92d2-43f0-8fdd-8b012243033f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:31.981884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b709a-92d2-43f0-8fdd-8b012243033f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:31.998878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0c3b709a-92d2-43f0-8fdd-8b012243033f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:32.000068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c3b709a-92d2-43f0-8fdd-8b012243033f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:32.001046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d93c4c-91da-41b5-b1c7-6669bafca373 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:32.029416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:32.075005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 503/1017] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:31 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:32.099061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b709a-92d2-43f0-8fdd-8b012243033f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:32.115853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d93c4c-91da-41b5-b1c7-6669bafca373 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:32.117994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:32.150716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:32.151755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:32.152310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:32.152536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:32.153746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:32.155234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:32.162206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 515/1018] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:32.163503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:32.210989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b709a-92d2-43f0-8fdd-8b012243033f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:32.248738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d332e7-e6a5-4bd7-b5fd-5ed184ad1df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:32.249443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d332e7-e6a5-4bd7-b5fd-5ed184ad1df1 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:32.249734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d332e7-e6a5-4bd7-b5fd-5ed184ad1df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:32.250015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d332e7-e6a5-4bd7-b5fd-5ed184ad1df1 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:32.250738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d332e7-e6a5-4bd7-b5fd-5ed184ad1df1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:32.252120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d332e7-e6a5-4bd7-b5fd-5ed184ad1df1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:32.268101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:32.270826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 504/1019] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:43:32 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:43:32.276943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872db34-9261-4dc3-9005-59326555b7d2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:32.324974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90c1c3-b6de-474c-81b0-7848cf64b9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:32.325474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90c1c3-b6de-474c-81b0-7848cf64b9f2 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:32.325758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90c1c3-b6de-474c-81b0-7848cf64b9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:32.326039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90c1c3-b6de-474c-81b0-7848cf64b9f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:32.326760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90c1c3-b6de-474c-81b0-7848cf64b9f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:32.328249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90c1c3-b6de-474c-81b0-7848cf64b9f2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:32.357877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 516/1020] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:43:32 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:33.176545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0812fa-a7ed-4c63-8ff9-53e0c933d517 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:33.177542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0812fa-a7ed-4c63-8ff9-53e0c933d517 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:33.178021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0812fa-a7ed-4c63-8ff9-53e0c933d517 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:33.179627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 505/1021] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:33.185172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f2b96-88eb-4ec7-af70-f39c1b3fc99a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:33.185609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f2b96-88eb-4ec7-af70-f39c1b3fc99a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:33.185942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f2b96-88eb-4ec7-af70-f39c1b3fc99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:33.203502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-598f2b96-88eb-4ec7-af70-f39c1b3fc99a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:33.204685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-598f2b96-88eb-4ec7-af70-f39c1b3fc99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:33.273373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 517/1022] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:33.295954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0812fa-a7ed-4c63-8ff9-53e0c933d517 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:33.392911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:33.421072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0812fa-a7ed-4c63-8ff9-53e0c933d517 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:33.458432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:33.459276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:33.459574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:33.459852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:33.470585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:33.479150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:33.480071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:33.480937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 506/1023] 10.222.0.91 () {56 vars in 1120 bytes} [Thu May 26 13:43:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:33.487610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f2b96-88eb-4ec7-af70-f39c1b3fc99a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:33.561756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:33.593869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f2b96-88eb-4ec7-af70-f39c1b3fc99a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:33.628743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:33.629439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:33.629719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:33.629990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:33.630730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:33.631039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:33.632396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:33.645172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 518/1024] 10.222.0.91 () {56 vars in 1115 bytes} [Thu May 26 13:43:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:33.646399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:33.756851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5379-3b8f-4aaf-9720-1022e50979a3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:33.790507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:33.791303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:33.791588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:33.791864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:33.794638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:33.804830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:33.806497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:33.823350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:33.824407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:33.825855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:33.825855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:33.825855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:33.825855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:33.826354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 507/1025] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:33.830266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:33.934697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1bf83-bab2-4d5a-97a8-d1cbee6d433a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:33.968988 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:33.969697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:33.969986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:33.970268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:33.971021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:33.971445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:33.972825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:33.974339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:33.986239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 519/1026] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:33.992112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:34.095460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eafdc-0a78-4129-823a-02a9db72ad31 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:34.130553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:34.131351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.131651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.131918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.139636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:34.141086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:34.155271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:34.156278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:34.157283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:34.158493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 508/1027] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:43:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:34.162323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:34.271512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6ebbd-3a41-413b-af2a-b4175ff9f3ab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:34.306813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95818ef0-8c79-4e68-ae76-49897144b705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:34.307489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95818ef0-8c79-4e68-ae76-49897144b705 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.307772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95818ef0-8c79-4e68-ae76-49897144b705 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.308044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95818ef0-8c79-4e68-ae76-49897144b705 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.308884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95818ef0-8c79-4e68-ae76-49897144b705 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:34.309211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95818ef0-8c79-4e68-ae76-49897144b705 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:34.310648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95818ef0-8c79-4e68-ae76-49897144b705 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:34.332517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 520/1028] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:43:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:34.334098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:34.393419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:34.446544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2c4e-403b-4111-b700-e8fe1c69586a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:34.484835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:34.485539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.485826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.486101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.555155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89639919-5007-4f75-9f44-fa0b68be7e14 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.555635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89639919-5007-4f75-9f44-fa0b68be7e14 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.556067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89639919-5007-4f75-9f44-fa0b68be7e14 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.557797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 521/1029] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:34.559844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:34.561428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:34.563104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305eae-97c1-4408-a30f-662a5be82a1a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.563396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305eae-97c1-4408-a30f-662a5be82a1a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.563669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305eae-97c1-4408-a30f-662a5be82a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.579733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-33305eae-97c1-4408-a30f-662a5be82a1a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:34.581467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33305eae-97c1-4408-a30f-662a5be82a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:34.613727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:34.636658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:34.659539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 522/1030] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:34.659716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb829fd-cd45-44b8-af2b-dc5480e86beb None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.659981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb829fd-cd45-44b8-af2b-dc5480e86beb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.660254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb829fd-cd45-44b8-af2b-dc5480e86beb None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.661533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 523/1031] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:34.666736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d031b-d3cc-4152-a136-cf3e640d26af None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.667089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d031b-d3cc-4152-a136-cf3e640d26af None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.667296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d031b-d3cc-4152-a136-cf3e640d26af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.677711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 509/1032] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:34 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:43:34.682455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:34.685748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f26d031b-d3cc-4152-a136-cf3e640d26af None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:34.686926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f26d031b-d3cc-4152-a136-cf3e640d26af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:34.770873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 524/1033] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:34.771711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d031b-d3cc-4152-a136-cf3e640d26af None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:34.802973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8825-0268-49e8-b017-b94d3f651162 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:34.836736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:34.837542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.837824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.838096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.838744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:34.840096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:34.851420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:34.852589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:34.853558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 510/1034] 10.222.0.91 () {56 vars in 1105 bytes} [Thu May 26 13:43:34 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:34.854661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:34.868418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:34.875039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d031b-d3cc-4152-a136-cf3e640d26af None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:34.888686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:34.909950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:34.910673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.910968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.911244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.921771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:34.923326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:34.942381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:34.943354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:34.944279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:34.944279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:34.944279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:34.944279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:34.944722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 525/1035] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:34.945751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:34.955112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384e787-9b52-41e8-893d-8811e91d9ee1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:34.988871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:34.989712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:34.989991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:34.990267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:34.991459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:34.992889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:34.994548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 511/1036] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:34.995303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.050940 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb298e-c47b-4c83-8c65-e9bef86addbe None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.083626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.084279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.084561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.084834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.085567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] RBAC: Adding query filter params (name=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:35.085870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.087846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.089258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:35.098484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 526/1037] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:35.098484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.116035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e8428-6482-4b22-b113-6c5067f37278 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.123589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:35.149502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.150169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.150452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.150728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.151440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:35.151747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.153357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.165431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 512/1038] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:35.166445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.206775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb5b9b-0226-48a3-8df2-bb1d3d37deda None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.245214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.245894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.246180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.246456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.247164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.248604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.269691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 527/1039] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:43:35 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:35.270492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.277876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d082087-6134-450f-95e7-494377a8b712 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.319426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.320108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.320651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.320933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.321801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:35.322112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.323531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.335147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 513/1040] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:35.336320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.340560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:35.398778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:35.424311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2dbaf-f4bc-4c7b-b146-92bf0f106a59 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.457971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692071c0-4924-4ebb-a17a-99394fa9a2a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.467207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.468304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.468799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.469297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.478586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.480377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:35.496783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:35.497811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:35.498729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 528/1041] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:35.500045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.500708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.500988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.501296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.502088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] RBAC: Adding query filter params (interface=public, service_id=231722103b2a47eb8cd2eef2599cd2be, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:35.502427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.503795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.503964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.510761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 514/1042] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:43:35 2022] GET /identity/v3/endpoints?service_id=231722103b2a47eb8cd2eef2599cd2be&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:35.516778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.589103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:35.621488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77288a21-80ac-4e93-90bd-562ef88a569d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.627448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227ae6e-84f2-4863-804d-40e27a8a6c09 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.659874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.660540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.660822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.661101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.661924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:35.662230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.663746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.665877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043e25d-9419-4da1-ad16-5d772c3f4ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.666786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043e25d-9419-4da1-ad16-5d772c3f4ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.667271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043e25d-9419-4da1-ad16-5d772c3f4ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.667855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043e25d-9419-4da1-ad16-5d772c3f4ee5 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.668785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043e25d-9419-4da1-ad16-5d772c3f4ee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.670380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043e25d-9419-4da1-ad16-5d772c3f4ee5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.676427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 529/1043] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:35.689197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 515/1044] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:43:35 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:43:35.690876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:35.694970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:35.805050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0ef2f-46a8-4a70-a9d0-bcc7072354ac None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:35.844387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bc4e0-4eb2-4033-92ea-c878d30e949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:35.845070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc4e0-4eb2-4033-92ea-c878d30e949f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:35.845388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc4e0-4eb2-4033-92ea-c878d30e949f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:35.845670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc4e0-4eb2-4033-92ea-c878d30e949f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:35.846571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bc4e0-4eb2-4033-92ea-c878d30e949f None admin] RBAC: Adding query filter params (role.id=5687582c149649509b22380e30cd1947, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=f579f2db041f455786a02dcf621619ed) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:35.846894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bc4e0-4eb2-4033-92ea-c878d30e949f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:35.848375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bc4e0-4eb2-4033-92ea-c878d30e949f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:35.854924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 530/1045] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:35 2022] GET /identity/v3/role_assignments?user.id=f579f2db041f455786a02dcf621619ed&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=5687582c149649509b22380e30cd1947 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:37.523026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9486-5997-46c4-bca4-c6d476077db0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:37.523571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9486-5997-46c4-bca4-c6d476077db0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:37.523571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9486-5997-46c4-bca4-c6d476077db0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:37.524899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 516/1046] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:37.530063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c3343-5d51-4a1b-8c44-d9b6c355c673 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:37.530499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c3343-5d51-4a1b-8c44-d9b6c355c673 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:37.530689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c3343-5d51-4a1b-8c44-d9b6c355c673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:37.548256 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-479c3343-5d51-4a1b-8c44-d9b6c355c673 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:37.550014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-479c3343-5d51-4a1b-8c44-d9b6c355c673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:37.612121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:37.619010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 531/1047] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:37.641610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b9486-5997-46c4-bca4-c6d476077db0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:37.746453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b9486-5997-46c4-bca4-c6d476077db0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:37.779699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:37.786478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02a946-9dce-4b17-842f-88377be62bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:37.787290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a946-9dce-4b17-842f-88377be62bec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:37.787570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a946-9dce-4b17-842f-88377be62bec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:37.787840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a946-9dce-4b17-842f-88377be62bec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:37.799370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02a946-9dce-4b17-842f-88377be62bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ed02a946-9dce-4b17-842f-88377be62bec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:37.807467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:37.808627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:37.810161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:37.813495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 517/1048] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:37.814679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c3343-5d51-4a1b-8c44-d9b6c355c673 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:37.920075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c3343-5d51-4a1b-8c44-d9b6c355c673 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:37.938161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301112-2778-4864-8808-a107e8b1b150 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:37.938487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301112-2778-4864-8808-a107e8b1b150 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:37.938746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301112-2778-4864-8808-a107e8b1b150 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:37.940550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 518/1049] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:37.947451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4b87-94a1-444a-9876-4093d290311a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:37.947789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4b87-94a1-444a-9876-4093d290311a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:37.947983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4b87-94a1-444a-9876-4093d290311a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:37.960049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:37.960725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:37.961012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:37.961317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:37.962074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:37.962385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:37.963764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:37.966138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f9ab4b87-94a1-444a-9876-4093d290311a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:37.967434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9ab4b87-94a1-444a-9876-4093d290311a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:37.978082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 532/1050] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:43:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:37.979109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.043056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.045292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 519/1051] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:38.066692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4b87-94a1-444a-9876-4093d290311a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.087862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e48d3-d010-450a-a47e-108ad2d16820 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.126042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa339d4b-d7ed-40eb-b983-072bec14293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.127220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339d4b-d7ed-40eb-b983-072bec14293b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.128123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339d4b-d7ed-40eb-b983-072bec14293b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.128572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339d4b-d7ed-40eb-b983-072bec14293b None admin] PATH_INFO: `/v3/users/975c5f63cddc4420bf58cddd12056e40` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.142615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa339d4b-d7ed-40eb-b983-072bec14293b None admin] RBAC: Authorizing `identity:get_user(user_id=975c5f63cddc4420bf58cddd12056e40)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.144643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa339d4b-d7ed-40eb-b983-072bec14293b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.150923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 533/1052] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:37 2022] GET /identity/v3/users/975c5f63cddc4420bf58cddd12056e40 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:38.150923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac134ed-71d7-4826-b258-1165b4357b76 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.150923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac134ed-71d7-4826-b258-1165b4357b76 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.150923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac134ed-71d7-4826-b258-1165b4357b76 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.150923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 534/1053] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:38.151782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac134ed-71d7-4826-b258-1165b4357b76 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.177281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4b87-94a1-444a-9876-4093d290311a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.202860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.215222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65b83c-84bd-44de-97ac-a56de62122cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.216027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65b83c-84bd-44de-97ac-a56de62122cf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.216314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65b83c-84bd-44de-97ac-a56de62122cf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.216590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65b83c-84bd-44de-97ac-a56de62122cf None admin] PATH_INFO: `/v3/services/image` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.217280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65b83c-84bd-44de-97ac-a56de62122cf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.218658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65b83c-84bd-44de-97ac-a56de62122cf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cc65b83c-84bd-44de-97ac-a56de62122cf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:38.230153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:38.231406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:38.232299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 520/1054] 10.222.0.91 () {56 vars in 1105 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:38.233177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ab002-3e5c-44f3-8f7a-90413fd7b905 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.233177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ab002-3e5c-44f3-8f7a-90413fd7b905 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.233332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ab002-3e5c-44f3-8f7a-90413fd7b905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.249539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cb9ab002-3e5c-44f3-8f7a-90413fd7b905 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:38.250768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cb9ab002-3e5c-44f3-8f7a-90413fd7b905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:38.262102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac134ed-71d7-4826-b258-1165b4357b76 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.279620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.297139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.297774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.298059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.298331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.319267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 521/1055] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:38.320102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ab002-3e5c-44f3-8f7a-90413fd7b905 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.368496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.369984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.420722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ab002-3e5c-44f3-8f7a-90413fd7b905 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.437976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.455864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.456523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.456803 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.457077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.457801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:38.458104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.459468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.464161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 522/1056] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:38.464920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.470889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 535/1057] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:38.471604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.489463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.564782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d28301-1831-47c7-8ee6-04e19511563a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.569564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682e564-c972-43c0-b281-7c007f0ff4c9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.598816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.599624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.599904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.600175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.603012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.603674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.603957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.604239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.604951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] RBAC: Adding query filter params (type=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:38.605374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.606674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.610596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:38.611607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 536/1058] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:38.612763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:38.619136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:38.620160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:38.621024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 523/1059] 10.222.0.91 () {56 vars in 1120 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:38.621782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.666117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.695499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.713881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae2d91-1089-482c-bf07-d9c1eda93009 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.725014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d2e8d-4e79-4bf8-b94f-4ff91cc018bd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.748200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.749068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.749380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.749660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.750846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] RBAC: Authorizing `identity:get_project(project_id=23366edf54c24010ab6fd9f0ae0bf129)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.752799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.754539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 537/1060] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:38.755300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.760849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.761762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.762064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.762343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.763083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.764447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.783981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 524/1061] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:43:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:38.786134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.867024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b818c-cb5d-4ceb-95fa-abdccc445dfa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.896306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.913861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea243e-8c20-48da-9d1a-97a0b9197247 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:38.917011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.917735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.918022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.918301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.919058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:38.919376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.920792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.931801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 538/1062] 10.222.0.91 () {56 vars in 1115 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:38.937554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:38.950116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:38.960922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:38.962224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:38.962714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:38.963189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:38.964644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=23366edf54c24010ab6fd9f0ae0bf129, user.id=975c5f63cddc4420bf58cddd12056e40) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:38.965133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:38.967350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:38.974582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 525/1063] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/role_assignments?user.id=975c5f63cddc4420bf58cddd12056e40&scope.project.id=23366edf54c24010ab6fd9f0ae0bf129&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:39.060234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f46e-c0dd-4c4b-8f6a-27412795527b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:39.100496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:39.101712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:39.102208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:39.102681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:39.116543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:39.118441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:39.138679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:39.139783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:39.140743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:39.140743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:39.140743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:39.140743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:39.140998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 539/1064] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:39.146416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:39.261392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04149b3b-68af-4f10-a2a2-41abb3a39d34 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:39.294772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:39.295436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:39.295715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:39.295983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:39.296956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:39.297282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:39.298644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:39.313966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 526/1065] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:39.319864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:39.383528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:39.428387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590108c-88f1-45e4-97e4-1bf7f96dd75c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:39.465607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:39.466411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:39.466698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:39.466972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:39.475178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:39.476682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:39.491351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:39.492382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:39.493446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 540/1066] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:43:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:39.498425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:39.517479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:39.602264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806c5c5-bbb5-4a71-8c54-96136d54f910 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:39.636810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:39.637428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:39.637710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:39.637981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:39.638817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:39.639122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:39.640543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:39.663176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 527/1067] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:43:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:39.664532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:39.768594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef07c-aa1a-4e68-be5d-e41dd06903f0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:39.789808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:39.806324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:39.807125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:39.807408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:39.807682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:39.818240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:39.819693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:39.837770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:39.838701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:39.844254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:39.844254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:39.844254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:39.844254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:39.844254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 541/1068] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:39.845974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:39.923802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:39.951099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257e65d-eedc-4a73-a094-c74de54b9270 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:39.988096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:39.988767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:39.989050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:39.989348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:39.990065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:39.990369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:39.991723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:40.007826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 528/1069] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:40.009327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:40.117998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796e58d-aa66-4a1f-b017-a567bb818f68 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:40.152791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:40.153623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:40.153913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:40.154192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:40.161688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:40.163140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:40.178410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:40.179379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:40.180304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:40.182621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 542/1070] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:40.185442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:40.289984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd2dcb-435c-43cd-9095-160c0cbd0b08 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:40.323372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e602412-0474-47fe-9908-14051685323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:40.324030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602412-0474-47fe-9908-14051685323c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:40.324313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602412-0474-47fe-9908-14051685323c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:40.324584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602412-0474-47fe-9908-14051685323c None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:40.325389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602412-0474-47fe-9908-14051685323c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:40.325699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602412-0474-47fe-9908-14051685323c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:40.327276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602412-0474-47fe-9908-14051685323c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:40.328919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:40.346720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 529/1071] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:40.348044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:40.380704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:40.454284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f30de2-d52b-47e0-a3c9-55872eaea1bb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:40.489634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a701a27-f70e-48cb-ad87-9ffc1a305ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:40.490588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701a27-f70e-48cb-ad87-9ffc1a305ae1 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:40.490875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701a27-f70e-48cb-ad87-9ffc1a305ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:40.491152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701a27-f70e-48cb-ad87-9ffc1a305ae1 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/f579f2db041f455786a02dcf621619ed/roles/5687582c149649509b22380e30cd1947` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:40.493438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a701a27-f70e-48cb-ad87-9ffc1a305ae1 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=f579f2db041f455786a02dcf621619ed, role_id=5687582c149649509b22380e30cd1947)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:40.494885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a701a27-f70e-48cb-ad87-9ffc1a305ae1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:40.513013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 543/1072] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:43:40 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/f579f2db041f455786a02dcf621619ed/roles/5687582c149649509b22380e30cd1947 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:43:41.063712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08c60-fa2e-4fa3-bccc-bba548c141a0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.064274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08c60-fa2e-4fa3-bccc-bba548c141a0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.064274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08c60-fa2e-4fa3-bccc-bba548c141a0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.065885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 530/1073] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:41.070898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd939d4-7e52-4ba0-8cd8-a433b8deabe2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.071339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd939d4-7e52-4ba0-8cd8-a433b8deabe2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.071496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd939d4-7e52-4ba0-8cd8-a433b8deabe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.088264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7bd939d4-7e52-4ba0-8cd8-a433b8deabe2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:41.089503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7bd939d4-7e52-4ba0-8cd8-a433b8deabe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:41.154615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02a38-ed49-484e-b249-3ffc5812b04a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.154910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02a38-ed49-484e-b249-3ffc5812b04a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.155212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02a38-ed49-484e-b249-3ffc5812b04a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.156714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 531/1074] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:41.161661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 544/1075] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:41.162357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4131288-7493-4f4b-be99-bec02b28a6ed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.162651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4131288-7493-4f4b-be99-bec02b28a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.162931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4131288-7493-4f4b-be99-bec02b28a6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.183492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4131288-7493-4f4b-be99-bec02b28a6ed None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:41.185537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4131288-7493-4f4b-be99-bec02b28a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:41.188499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd939d4-7e52-4ba0-8cd8-a433b8deabe2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:41.213008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:41.258648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 532/1076] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:41.280075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4131288-7493-4f4b-be99-bec02b28a6ed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:41.296965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd939d4-7e52-4ba0-8cd8-a433b8deabe2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:41.332919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:41.333539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.333819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.334088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.386404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4131288-7493-4f4b-be99-bec02b28a6ed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:41.405859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:41.407874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:41.411646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:41.422771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:41.423586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.423851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.424642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.432913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:41.437362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:41.447049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:41.448088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:41.448951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 533/1077] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:41.454088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:41.527609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 545/1078] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:43:41.533015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:41.571783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc88c08-1084-4a1a-abc9-63847d3783e5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:41.607220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:41.607818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.608103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.608383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.609132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:41.609468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:41.610837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:41.621234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 534/1079] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:43:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:41.627779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:41.645685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a310-7e98-4bdf-b151-7108caab5f9e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:41.668789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:41.681202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:41.683097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:41.684046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.684376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.684659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.695741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:41.697212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:41.716105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:41.717078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:41.718148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:41.718148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:41.718148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:41.718148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:41.718148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 546/1080] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:41.723662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:41.746292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda5c5f-f655-4d06-a565-792e3b40c923 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:41.785288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:41.786276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.786617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.786963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] PATH_INFO: `/v3/users/975c5f63cddc4420bf58cddd12056e40` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.788240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=975c5f63cddc4420bf58cddd12056e40)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:41.789804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:41.791449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 535/1081] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:41 2022] GET /identity/v3/users/975c5f63cddc4420bf58cddd12056e40 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:41.797328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:41.836485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1ec87-6bf2-4218-83aa-5540bebc403d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:41.872421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:41.873002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.873330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.873608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.874327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:41.874630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:41.875994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:41.897176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 547/1082] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:41.915507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca16ab-ad31-4907-8d0c-f6b77caef7b3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:41.933418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:41.957208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec73c8-0c5f-4a7b-8491-3455bd0ffa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:41.958198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec73c8-0c5f-4a7b-8491-3455bd0ffa31 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:41.958402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec73c8-0c5f-4a7b-8491-3455bd0ffa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:41.958684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec73c8-0c5f-4a7b-8491-3455bd0ffa31 None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:41.959937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec73c8-0c5f-4a7b-8491-3455bd0ffa31 None admin] RBAC: Authorizing `identity:get_project(project_id=23366edf54c24010ab6fd9f0ae0bf129)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:41.961451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec73c8-0c5f-4a7b-8491-3455bd0ffa31 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:41.963248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 536/1083] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:43:41 2022] GET /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:41.970252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:42.009356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:42.087350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede1e3d-4de9-4bbf-9e4a-5a6a8a3ebf66 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:42.129869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be738f59-181c-43fb-a57a-b39f4bed9b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:42.130702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be738f59-181c-43fb-a57a-b39f4bed9b44 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:42.131041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be738f59-181c-43fb-a57a-b39f4bed9b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:42.131268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be738f59-181c-43fb-a57a-b39f4bed9b44 None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129/users/975c5f63cddc4420bf58cddd12056e40/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:42.133384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be738f59-181c-43fb-a57a-b39f4bed9b44 None admin] RBAC: Authorizing `identity:create_grant(project_id=23366edf54c24010ab6fd9f0ae0bf129, user_id=975c5f63cddc4420bf58cddd12056e40, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:42.134837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be738f59-181c-43fb-a57a-b39f4bed9b44 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:42.151084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 548/1084] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:43:41 2022] PUT /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129/users/975c5f63cddc4420bf58cddd12056e40/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:43:42.732996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956ea3c-87f2-46f3-b665-e98902572342 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:42.733590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956ea3c-87f2-46f3-b665-e98902572342 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:42.733590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956ea3c-87f2-46f3-b665-e98902572342 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:42.735125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 537/1085] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:42.740768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2edca9-c5c2-4100-9db0-39b77954640c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:42.741168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2edca9-c5c2-4100-9db0-39b77954640c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:42.741526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2edca9-c5c2-4100-9db0-39b77954640c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:42.757982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7b2edca9-c5c2-4100-9db0-39b77954640c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:42.759169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7b2edca9-c5c2-4100-9db0-39b77954640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:42.827387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 549/1086] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:42.851735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956ea3c-87f2-46f3-b665-e98902572342 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:42.951724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956ea3c-87f2-46f3-b665-e98902572342 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:42.985636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:42.986364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:42.986673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:42.986949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:42.996834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:43.005010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:43.005986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:43.006873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 538/1087] 10.222.0.91 () {56 vars in 1120 bytes} [Thu May 26 13:43:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:43.012301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2edca9-c5c2-4100-9db0-39b77954640c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:43.039709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:43.112973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2edca9-c5c2-4100-9db0-39b77954640c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:43.145444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:43.146105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:43.146384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:43.146656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:43.147393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:43.147695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:43.149070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:43.156779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 550/1088] 10.222.0.91 () {56 vars in 1115 bytes} [Thu May 26 13:43:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:43.163365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:43.214128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:43.276219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52256f1d-4314-459f-862f-8630246bc60a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:43.311873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:43.312671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:43.312952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:43.313251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:43.325114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:43.326599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:43.344687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:43.345791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:43.347162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:43.347162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:43.347162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:43.347162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:43.347162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 539/1089] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:43.351692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:43.442880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:43.460029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dfde3-d32b-48b1-8b92-25f867960c7b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:43.493401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:43.494071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:43.494353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:43.494627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:43.495521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:43.495843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:43.497251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:43.510875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 551/1090] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:43.516695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:43.617798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:43.625826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298e6c1-858b-4314-91cc-9b2225a49422 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:43.663841 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:43.664640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:43.664925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:43.665290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:43.674525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:43.676258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:43:43.691310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:43.692390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:43.693419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 540/1091] 10.222.0.91 () {56 vars in 1093 bytes} [Thu May 26 13:43:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:43.698641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:43.813476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b8c18-4fb9-473b-bf1b-4bca9f12eca7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:43.851143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:43.851830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:43.852113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:43.852392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:43.853261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:43.853576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:43.855023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:43.874553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 552/1092] 10.222.0.91 () {56 vars in 1133 bytes} [Thu May 26 13:43:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:43.880672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:43.989009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c762442-0d7d-4048-9d33-dd58de305b49 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.022870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b284325-6faf-4bd1-bcee-5b0c05fd13db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.023520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284325-6faf-4bd1-bcee-5b0c05fd13db None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.023805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284325-6faf-4bd1-bcee-5b0c05fd13db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.024082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284325-6faf-4bd1-bcee-5b0c05fd13db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.050671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:44.097814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b284325-6faf-4bd1-bcee-5b0c05fd13db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:44.099258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b284325-6faf-4bd1-bcee-5b0c05fd13db None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:44.213084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 541/1093] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:43:44.219096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.223689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:44.323656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39dc79-3b24-476b-adc9-8e2c1843f265 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.336392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71060-cc7a-4a17-9e32-bf238cc683a2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.336697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71060-cc7a-4a17-9e32-bf238cc683a2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.336944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71060-cc7a-4a17-9e32-bf238cc683a2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.338516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 542/1094] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:43:44.343610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57a802-107b-4f14-a972-a3f1d036ee25 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.343890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57a802-107b-4f14-a972-a3f1d036ee25 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.344165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57a802-107b-4f14-a972-a3f1d036ee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.361948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0f57a802-107b-4f14-a972-a3f1d036ee25 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:44.363155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f57a802-107b-4f14-a972-a3f1d036ee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:44.365929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.367272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.367272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.367272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.377996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:44.379472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:43:44.398087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:44.399165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:44.400073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:43:44.400073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:43:44.400073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:43:44.400073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:44.400073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 553/1095] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:44.405119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.421663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:44.441870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 543/1096] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:43:44.468948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57a802-107b-4f14-a972-a3f1d036ee25 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.474849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:44.525046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a22464-f468-4d69-9840-56a2bc1c1dcf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.571297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.571999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.572281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.572565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.573354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:44.573670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:44.575092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:44.587899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57a802-107b-4f14-a972-a3f1d036ee25 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.588141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 554/1097] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:44.595388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.631192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.631995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.632275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.632550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.644235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:43:44.654257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:44.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:43:44.657813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 544/1098] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:43:44.666008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.713960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:44.715311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84674a-9f97-429a-976b-8ceec047d3b8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.732835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:44.751609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.753007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.753329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.753614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.760293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:44.761796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:44.774377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614b9f8-4edb-4472-bdfa-35e7dfb4194d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:43:44.776686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:43:44.778313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:43:44.780189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:43:44.781500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 555/1099] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:43:44.784081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.824335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f43692-fe54-4715-862a-713590612422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.825293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43692-fe54-4715-862a-713590612422 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.825758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43692-fe54-4715-862a-713590612422 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.826148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43692-fe54-4715-862a-713590612422 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.827416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f43692-fe54-4715-862a-713590612422 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:44.827932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f43692-fe54-4715-862a-713590612422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:44.830846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f43692-fe54-4715-862a-713590612422 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:44.839534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 545/1100] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:44.845193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f43692-fe54-4715-862a-713590612422 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.906266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00361f-55be-466b-b58d-76f6cf7a17b1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.941237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.941708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.941991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.942270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.943042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:44.943349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:44.944783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:44.947154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f43692-fe54-4715-862a-713590612422 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:44.954510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:44.957009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 556/1101] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:44.962756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.982641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:44.983572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:44.983800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:44.984076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] PATH_INFO: `/v3/users/975c5f63cddc4420bf58cddd12056e40` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:44.985293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=975c5f63cddc4420bf58cddd12056e40)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:44.986732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:44.988350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 546/1102] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/users/975c5f63cddc4420bf58cddd12056e40 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:44.993857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:44.996514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:45.068310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f7c13-11ed-4a45-aacc-73c261d1c334 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:45.096714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42eb01-76cc-400e-8078-5a1c8bb84aa5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:45.101453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:45.102109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:45.102390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:45.102659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:45.103478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] RBAC: Adding query filter params (role.id=5687582c149649509b22380e30cd1947, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=f579f2db041f455786a02dcf621619ed) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:45.103786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:45.105241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:45.112482 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 557/1103] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/role_assignments?user.id=f579f2db041f455786a02dcf621619ed&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=5687582c149649509b22380e30cd1947 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:45.131217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:45.131891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:45.132170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:45.132439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:45.182416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:45.210788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:45.212278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:45.318976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 547/1104] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:43:45.324900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:45.428154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37115a5a-8ec9-4aea-a773-e87e2da1775c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:45.429890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:45.464385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9cb3f-0226-44cb-935e-f68582834344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:45.465287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9cb3f-0226-44cb-935e-f68582834344 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:45.465576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9cb3f-0226-44cb-935e-f68582834344 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:45.465855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9cb3f-0226-44cb-935e-f68582834344 None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:45.467041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9cb3f-0226-44cb-935e-f68582834344 None admin] RBAC: Authorizing `identity:get_project(project_id=23366edf54c24010ab6fd9f0ae0bf129)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:45.468485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9cb3f-0226-44cb-935e-f68582834344 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:45.470220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 558/1105] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:43:45 2022] GET /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:45.476943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:45.556593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:45.582200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1e7dc-3e55-43a6-966a-218afb0fe535 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:45.616619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46daeae9-312e-494f-a2ec-b1c526974713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:45.617299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daeae9-312e-494f-a2ec-b1c526974713 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:45.617589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daeae9-312e-494f-a2ec-b1c526974713 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:45.617859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daeae9-312e-494f-a2ec-b1c526974713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:45.618685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daeae9-312e-494f-a2ec-b1c526974713 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=23366edf54c24010ab6fd9f0ae0bf129, user.id=975c5f63cddc4420bf58cddd12056e40) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:45.618995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daeae9-312e-494f-a2ec-b1c526974713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:45.620407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daeae9-312e-494f-a2ec-b1c526974713 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:45.626602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 548/1106] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:45 2022] GET /identity/v3/role_assignments?user.id=975c5f63cddc4420bf58cddd12056e40&scope.project.id=23366edf54c24010ab6fd9f0ae0bf129&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:47.893598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfca3f-c4df-4fc4-8ec0-68572a568ba9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:47.894133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfca3f-c4df-4fc4-8ec0-68572a568ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:47.894133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfca3f-c4df-4fc4-8ec0-68572a568ba9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:47.895549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 559/1107] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:47.900752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dec7b-9167-4226-b093-f30ef4431f56 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:47.901187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dec7b-9167-4226-b093-f30ef4431f56 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:47.901336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dec7b-9167-4226-b093-f30ef4431f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:47.918330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e49dec7b-9167-4226-b093-f30ef4431f56 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:47.919511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e49dec7b-9167-4226-b093-f30ef4431f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:47.984641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 549/1108] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:48.008934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfca3f-c4df-4fc4-8ec0-68572a568ba9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:48.108197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfca3f-c4df-4fc4-8ec0-68572a568ba9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:48.147082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddffcb1-f181-4eca-a0d7-74c29204e40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:48.147796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddffcb1-f181-4eca-a0d7-74c29204e40d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:48.148083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddffcb1-f181-4eca-a0d7-74c29204e40d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:48.148360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddffcb1-f181-4eca-a0d7-74c29204e40d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:48.149569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddffcb1-f181-4eca-a0d7-74c29204e40d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:48.151011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddffcb1-f181-4eca-a0d7-74c29204e40d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:48.152630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 560/1109] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:48.159193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dec7b-9167-4226-b093-f30ef4431f56 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:48.223455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:48.255648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dec7b-9167-4226-b093-f30ef4431f56 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:48.289831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3babaf-7de9-470e-bc32-b5a04c8797f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:48.290503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3babaf-7de9-470e-bc32-b5a04c8797f3 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:48.290787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3babaf-7de9-470e-bc32-b5a04c8797f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:48.291064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3babaf-7de9-470e-bc32-b5a04c8797f3 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:48.291768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3babaf-7de9-470e-bc32-b5a04c8797f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:48.293221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3babaf-7de9-470e-bc32-b5a04c8797f3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:48.312826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 550/1110] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:43:48 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:43:50.568583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795211-2dac-4af2-987f-42e8e6b5af13 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:50.569087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795211-2dac-4af2-987f-42e8e6b5af13 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:50.569087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795211-2dac-4af2-987f-42e8e6b5af13 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:50.570456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 561/1111] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:50.575900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c62539-6862-4a80-ad0d-609990e95413 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:50.576350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c62539-6862-4a80-ad0d-609990e95413 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:50.576445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c62539-6862-4a80-ad0d-609990e95413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:50.593024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-86c62539-6862-4a80-ad0d-609990e95413 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:50.594242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-86c62539-6862-4a80-ad0d-609990e95413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:50.659696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 551/1112] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:50.683568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795211-2dac-4af2-987f-42e8e6b5af13 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:50.719307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:50.785948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795211-2dac-4af2-987f-42e8e6b5af13 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:50.819732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:50.820524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:50.820813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:50.821093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] PATH_INFO: `/v3/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:50.822581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] RBAC: Authorizing `identity:get_role(role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:50.824022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:50.825422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 562/1113] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:50 2022] GET /identity/v3/roles/33a3e99f754a4bc69e665330de985b76 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:50.831189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c62539-6862-4a80-ad0d-609990e95413 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:50.883785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:50.929229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c62539-6862-4a80-ad0d-609990e95413 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:50.962246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:50.963110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:50.963386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:50.963653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] PATH_INFO: `/v3/users/09b29444a2314dd98e04fd9bad13546a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:50.977059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] RBAC: Authorizing `identity:get_user(user_id=09b29444a2314dd98e04fd9bad13546a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:50.978539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:50.980283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 552/1114] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:50 2022] GET /identity/v3/users/09b29444a2314dd98e04fd9bad13546a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:50.986022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:51.082818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807fe9d-1e4c-4504-9021-285456f7403c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:51.084978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:51.115322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:51.115959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:51.116238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:51.116514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:51.180669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:51.182134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:51.276250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 563/1115] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:51.282224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:51.379200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0bbfa-6d87-4539-8e56-6e5f9d514471 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:51.391579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:51.412852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d44bc-e9ab-4ce2-b520-5092fcb1b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:51.413721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d44bc-e9ab-4ce2-b520-5092fcb1b613 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:51.414001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d44bc-e9ab-4ce2-b520-5092fcb1b613 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:51.414272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d44bc-e9ab-4ce2-b520-5092fcb1b613 None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:51.415423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d44bc-e9ab-4ce2-b520-5092fcb1b613 None admin] RBAC: Authorizing `identity:get_project(project_id=23366edf54c24010ab6fd9f0ae0bf129)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:51.416838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d44bc-e9ab-4ce2-b520-5092fcb1b613 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:51.418537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 553/1116] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:43:51 2022] GET /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:51.424796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:51.427798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:51.523935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb8caf-0f01-4502-b458-8b93ba1cb7c5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:51.555982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d9790-c249-4b41-9e3d-06164af1ada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:51.556913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9790-c249-4b41-9e3d-06164af1ada1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:51.557218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9790-c249-4b41-9e3d-06164af1ada1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:51.557492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9790-c249-4b41-9e3d-06164af1ada1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:51.558312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d9790-c249-4b41-9e3d-06164af1ada1 None admin] RBAC: Adding query filter params (role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=23366edf54c24010ab6fd9f0ae0bf129, user.id=09b29444a2314dd98e04fd9bad13546a) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:51.558618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d9790-c249-4b41-9e3d-06164af1ada1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:51.560043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d9790-c249-4b41-9e3d-06164af1ada1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:51.565790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 564/1117] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:51 2022] GET /identity/v3/role_assignments?user.id=09b29444a2314dd98e04fd9bad13546a&scope.project.id=23366edf54c24010ab6fd9f0ae0bf129&role.id=33a3e99f754a4bc69e665330de985b76 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:43:53.719767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eeef1-0d5d-4883-9f4e-43deb8dee004 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:53.720289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eeef1-0d5d-4883-9f4e-43deb8dee004 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:53.720382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eeef1-0d5d-4883-9f4e-43deb8dee004 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:53.722225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 554/1118] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:53.727316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d57aa-706b-4944-937a-5e404177a5b9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:53.727742 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d57aa-706b-4944-937a-5e404177a5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:53.727878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d57aa-706b-4944-937a-5e404177a5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:53.745602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0a5d57aa-706b-4944-937a-5e404177a5b9 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:53.746772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0a5d57aa-706b-4944-937a-5e404177a5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:53.806730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:53.812364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 565/1119] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:53.834743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5eeef1-0d5d-4883-9f4e-43deb8dee004 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:53.934247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5eeef1-0d5d-4883-9f4e-43deb8dee004 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:53.966274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:53.967127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:53.967443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:53.967716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] PATH_INFO: `/v3/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:53.969201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] RBAC: Authorizing `identity:get_role(role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:53.971143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:53.972458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 555/1120] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:53 2022] GET /identity/v3/roles/33a3e99f754a4bc69e665330de985b76 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:53.978199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d57aa-706b-4944-937a-5e404177a5b9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:54.071711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d57aa-706b-4944-937a-5e404177a5b9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:54.104452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:54.105341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:54.105619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:54.105889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] PATH_INFO: `/v3/users/09b29444a2314dd98e04fd9bad13546a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:54.107062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=09b29444a2314dd98e04fd9bad13546a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:54.108486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:54.110090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 566/1121] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:53 2022] GET /identity/v3/users/09b29444a2314dd98e04fd9bad13546a => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:54.115776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:54.163923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:54.214676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb892a-0ae1-4cd6-8052-9f570179dcf1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:54.247875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae180b53-514e-4693-ba04-b8a6b1822189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:54.248663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae180b53-514e-4693-ba04-b8a6b1822189 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:54.248943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae180b53-514e-4693-ba04-b8a6b1822189 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:54.249247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae180b53-514e-4693-ba04-b8a6b1822189 None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:54.250402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae180b53-514e-4693-ba04-b8a6b1822189 None admin] RBAC: Authorizing `identity:get_project(project_id=23366edf54c24010ab6fd9f0ae0bf129)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:54.252369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae180b53-514e-4693-ba04-b8a6b1822189 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:54.254078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 556/1122] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:43:54 2022] GET /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:54.259979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:54.324008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:54.358086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5155cdc-21a8-4db1-8b33-ad3022212eb7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:54.392430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef795c9-6bfa-48be-baaf-ab1e9968ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:54.393345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef795c9-6bfa-48be-baaf-ab1e9968ca1b None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:54.393632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef795c9-6bfa-48be-baaf-ab1e9968ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:54.393909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef795c9-6bfa-48be-baaf-ab1e9968ca1b None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129/users/09b29444a2314dd98e04fd9bad13546a/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:54.396095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef795c9-6bfa-48be-baaf-ab1e9968ca1b None admin] RBAC: Authorizing `identity:create_grant(project_id=23366edf54c24010ab6fd9f0ae0bf129, user_id=09b29444a2314dd98e04fd9bad13546a, role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:54.397568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef795c9-6bfa-48be-baaf-ab1e9968ca1b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:54.409725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 567/1123] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:43:54 2022] PUT /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129/users/09b29444a2314dd98e04fd9bad13546a/roles/33a3e99f754a4bc69e665330de985b76 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:43:56.580336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a72c5c-f544-45ec-a60b-a6e68cc212da None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:56.580844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a72c5c-f544-45ec-a60b-a6e68cc212da None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:56.581069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a72c5c-f544-45ec-a60b-a6e68cc212da None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:56.582486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 557/1124] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:56.587861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9443380-8465-4d94-a613-25d57c3f4a72 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:56.588260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9443380-8465-4d94-a613-25d57c3f4a72 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:56.588427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9443380-8465-4d94-a613-25d57c3f4a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:56.605501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d9443380-8465-4d94-a613-25d57c3f4a72 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:56.606702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d9443380-8465-4d94-a613-25d57c3f4a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:56.671875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 568/1125] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:56.695359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a72c5c-f544-45ec-a60b-a6e68cc212da None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:56.793353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a72c5c-f544-45ec-a60b-a6e68cc212da None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:56.809242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:56.827000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:56.827847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:56.828124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:56.828394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] PATH_INFO: `/v3/roles/33a3e99f754a4bc69e665330de985b76` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:56.830482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] RBAC: Authorizing `identity:get_role(role_id=33a3e99f754a4bc69e665330de985b76)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:56.831793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:56.833164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 558/1126] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:56 2022] GET /identity/v3/roles/33a3e99f754a4bc69e665330de985b76 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:56.838995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9443380-8465-4d94-a613-25d57c3f4a72 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:56.893505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:56.941930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9443380-8465-4d94-a613-25d57c3f4a72 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:56.975882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:56.976743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:56.977027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:56.977337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] PATH_INFO: `/v3/users/09b29444a2314dd98e04fd9bad13546a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:56.978512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] RBAC: Authorizing `identity:get_user(user_id=09b29444a2314dd98e04fd9bad13546a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:56.979972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:56.981623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 569/1127] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:43:56 2022] GET /identity/v3/users/09b29444a2314dd98e04fd9bad13546a => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:56.987431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:57.084462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed422-5fc0-44b5-abb0-0c3518ac66c5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:57.118850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:57.119495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:57.119779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:57.120057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:57.169590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:57.186058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:57.187508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:57.282951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 559/1128] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:43:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:43:57.288782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:57.384344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d76ebb-b627-4c51-a3c8-afd4737b2fda None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:57.406656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:57.417257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9820b7c-eeff-47de-904b-b525f15d44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:57.418104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9820b7c-eeff-47de-904b-b525f15d44e4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:57.418384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9820b7c-eeff-47de-904b-b525f15d44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:57.418656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9820b7c-eeff-47de-904b-b525f15d44e4 None admin] PATH_INFO: `/v3/projects/23366edf54c24010ab6fd9f0ae0bf129` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:57.419809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9820b7c-eeff-47de-904b-b525f15d44e4 None admin] RBAC: Authorizing `identity:get_project(project_id=23366edf54c24010ab6fd9f0ae0bf129)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:57.421524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9820b7c-eeff-47de-904b-b525f15d44e4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:57.423170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 570/1129] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:43:57 2022] GET /identity/v3/projects/23366edf54c24010ab6fd9f0ae0bf129 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:57.429857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:57.512125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:57.528560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdac17b-46a6-495f-a855-909669b8ba98 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:57.562559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7564a65-2f39-4abb-bfbd-8eded4c84c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:57.563208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564a65-2f39-4abb-bfbd-8eded4c84c8b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:57.563489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564a65-2f39-4abb-bfbd-8eded4c84c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:57.563762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564a65-2f39-4abb-bfbd-8eded4c84c8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:57.564637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564a65-2f39-4abb-bfbd-8eded4c84c8b None admin] RBAC: Adding query filter params (role.id=33a3e99f754a4bc69e665330de985b76, scope.project.id=23366edf54c24010ab6fd9f0ae0bf129, user.id=09b29444a2314dd98e04fd9bad13546a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:43:57.564951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564a65-2f39-4abb-bfbd-8eded4c84c8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:57.566417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564a65-2f39-4abb-bfbd-8eded4c84c8b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:57.572141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 560/1130] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:43:57 2022] GET /identity/v3/role_assignments?user.id=09b29444a2314dd98e04fd9bad13546a&scope.project.id=23366edf54c24010ab6fd9f0ae0bf129&role.id=33a3e99f754a4bc69e665330de985b76 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:43:59.789566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf180b6b-6709-4947-a6f3-9ffc6b4079c7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:59.790119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf180b6b-6709-4947-a6f3-9ffc6b4079c7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:59.790119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf180b6b-6709-4947-a6f3-9ffc6b4079c7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:59.791493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 571/1131] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:59.796987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb5be-f5cb-45fd-8ad9-e65fc32dff59 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:59.797426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb5be-f5cb-45fd-8ad9-e65fc32dff59 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:59.797580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb5be-f5cb-45fd-8ad9-e65fc32dff59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:59.814403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5ccdb5be-f5cb-45fd-8ad9-e65fc32dff59 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:59.815586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ccdb5be-f5cb-45fd-8ad9-e65fc32dff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:59.879003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 561/1132] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:43:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:43:59.902625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf180b6b-6709-4947-a6f3-9ffc6b4079c7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:00.003497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf180b6b-6709-4947-a6f3-9ffc6b4079c7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:00.036500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010a658-995f-41a4-8c4d-af1acd3a9d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:00.037313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010a658-995f-41a4-8c4d-af1acd3a9d5e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:00.037601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010a658-995f-41a4-8c4d-af1acd3a9d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:00.037881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010a658-995f-41a4-8c4d-af1acd3a9d5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:00.039056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010a658-995f-41a4-8c4d-af1acd3a9d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:00.040547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010a658-995f-41a4-8c4d-af1acd3a9d5e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:00.042210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 572/1133] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:43:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:00.048176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdb5be-f5cb-45fd-8ad9-e65fc32dff59 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:00.123167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:00.144420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdb5be-f5cb-45fd-8ad9-e65fc32dff59 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:00.177244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd75c8-eb1e-4095-995c-5ce891c2f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:00.177909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd75c8-eb1e-4095-995c-5ce891c2f4d4 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:00.178191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd75c8-eb1e-4095-995c-5ce891c2f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:00.178467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd75c8-eb1e-4095-995c-5ce891c2f4d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:00.179172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd75c8-eb1e-4095-995c-5ce891c2f4d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:00.180591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd75c8-eb1e-4095-995c-5ce891c2f4d4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:00.207665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 562/1134] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:44:00 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:02.423867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f30be9-93e1-437b-b76f-8f2563d6e588 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:02.424418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f30be9-93e1-437b-b76f-8f2563d6e588 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:02.424418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f30be9-93e1-437b-b76f-8f2563d6e588 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:02.425829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 573/1135] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:02.431091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65c767-3da4-4276-829f-4082069910ef None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:02.431518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65c767-3da4-4276-829f-4082069910ef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:02.431655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65c767-3da4-4276-829f-4082069910ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:02.448270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5a65c767-3da4-4276-829f-4082069910ef None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:02.449419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a65c767-3da4-4276-829f-4082069910ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:02.515869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 563/1136] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:02.541410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f30be9-93e1-437b-b76f-8f2563d6e588 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:02.602657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:02.642826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f30be9-93e1-437b-b76f-8f2563d6e588 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:02.675779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9783e6f-7a24-4419-8344-51a59dd4f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:02.676555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9783e6f-7a24-4419-8344-51a59dd4f5de None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:02.676840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9783e6f-7a24-4419-8344-51a59dd4f5de None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:02.677115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9783e6f-7a24-4419-8344-51a59dd4f5de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:02.678319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9783e6f-7a24-4419-8344-51a59dd4f5de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:02.679761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9783e6f-7a24-4419-8344-51a59dd4f5de None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:02.681382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 574/1137] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:44:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:02.687082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65c767-3da4-4276-829f-4082069910ef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:02.743683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:02.783982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65c767-3da4-4276-829f-4082069910ef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:02.816798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b6a8-4e81-4ad7-9328-d0c9ee34a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:02.817490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b6a8-4e81-4ad7-9328-d0c9ee34a86f None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:02.817777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b6a8-4e81-4ad7-9328-d0c9ee34a86f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:02.818056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b6a8-4e81-4ad7-9328-d0c9ee34a86f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:02.818757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6b6a8-4e81-4ad7-9328-d0c9ee34a86f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:02.820191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6b6a8-4e81-4ad7-9328-d0c9ee34a86f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:02.840421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 564/1138] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:44:02 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:05.090188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e58bf-59b3-449a-9292-80de0c8c3e99 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:05.090753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e58bf-59b3-449a-9292-80de0c8c3e99 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:05.090753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e58bf-59b3-449a-9292-80de0c8c3e99 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:05.092102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 575/1139] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:05.097949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e68a1-97f9-4ce1-a37a-0bbcd8e5dff1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:05.098325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e68a1-97f9-4ce1-a37a-0bbcd8e5dff1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:05.098512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e68a1-97f9-4ce1-a37a-0bbcd8e5dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:05.115819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-879e68a1-97f9-4ce1-a37a-0bbcd8e5dff1 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:05.117446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-879e68a1-97f9-4ce1-a37a-0bbcd8e5dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:05.183260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 565/1140] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:05.206492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e58bf-59b3-449a-9292-80de0c8c3e99 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:05.305372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e58bf-59b3-449a-9292-80de0c8c3e99 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:05.330942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:05.338520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:05.339310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:05.339585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:05.339858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:05.350378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:44:05.357853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:44:05.358999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:44:05.359962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 576/1141] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:44:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:44:05.365130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e68a1-97f9-4ce1-a37a-0bbcd8e5dff1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:05.410296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:05.461724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e68a1-97f9-4ce1-a37a-0bbcd8e5dff1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:05.494147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:05.494809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:05.495089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:05.495361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:05.496090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:05.496670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:05.498060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:05.505820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 566/1142] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:05.511614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:05.624564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0908-fbda-4f85-a21d-d0659d946792 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:05.659219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:05.660065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:05.660344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:05.660613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] PATH_INFO: `/v3/users/20a6f28828784a77bf3c66a6fadea7ee` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:05.673613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] RBAC: Authorizing `identity:get_user(user_id=20a6f28828784a77bf3c66a6fadea7ee)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:05.675049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:05.676659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 577/1143] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:44:05 2022] GET /identity/v3/users/20a6f28828784a77bf3c66a6fadea7ee => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:05.682482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:05.780655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4975d9a-60b7-4678-aa05-041bb157a64c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:05.789236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:05.814393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:05.815036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:05.815323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:05.815603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:05.880442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:05.881860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:05.982699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 567/1144] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:44:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:44:05.988603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:06.035190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:06.087281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fb8c9-b1f3-4644-bccb-6875433922b0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:06.120273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c159e-3e25-48d3-9a0b-17f39ac5d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:06.121119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c159e-3e25-48d3-9a0b-17f39ac5d02d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:06.121434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c159e-3e25-48d3-9a0b-17f39ac5d02d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:06.121713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c159e-3e25-48d3-9a0b-17f39ac5d02d None admin] PATH_INFO: `/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:06.122887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c159e-3e25-48d3-9a0b-17f39ac5d02d None admin] RBAC: Authorizing `identity:get_project(project_id=1808f68ba63747f5b6e0cfa8d8d9447e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:06.124326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c159e-3e25-48d3-9a0b-17f39ac5d02d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:06.126029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 578/1145] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:44:05 2022] GET /identity/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:06.132696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:06.142807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:06.231432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f0437-3073-41b5-8c43-8ae8af34d443 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:06.265240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422b708-6f0e-483f-87d1-c7be1a21a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:06.265900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b708-6f0e-483f-87d1-c7be1a21a5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:06.266185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b708-6f0e-483f-87d1-c7be1a21a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:06.266460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b708-6f0e-483f-87d1-c7be1a21a5d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:06.267286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422b708-6f0e-483f-87d1-c7be1a21a5d3 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=1808f68ba63747f5b6e0cfa8d8d9447e, user.id=20a6f28828784a77bf3c66a6fadea7ee) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:06.267593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422b708-6f0e-483f-87d1-c7be1a21a5d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:06.269033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422b708-6f0e-483f-87d1-c7be1a21a5d3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:06.274387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 568/1146] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:44:06 2022] GET /identity/v3/role_assignments?user.id=20a6f28828784a77bf3c66a6fadea7ee&scope.project.id=1808f68ba63747f5b6e0cfa8d8d9447e&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:08.448400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ed0a8-1c4a-4c41-92f4-474481b06198 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:08.448955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ed0a8-1c4a-4c41-92f4-474481b06198 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:08.448955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ed0a8-1c4a-4c41-92f4-474481b06198 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:08.450272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 579/1147] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:08.456125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8047dd-4c59-4631-b2a9-a0bf35f3211e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:08.456525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8047dd-4c59-4631-b2a9-a0bf35f3211e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:08.456671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8047dd-4c59-4631-b2a9-a0bf35f3211e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:08.473534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2c8047dd-4c59-4631-b2a9-a0bf35f3211e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:08.474694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2c8047dd-4c59-4631-b2a9-a0bf35f3211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:08.538599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 569/1148] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:08.560611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ed0a8-1c4a-4c41-92f4-474481b06198 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:08.673728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ed0a8-1c4a-4c41-92f4-474481b06198 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:08.691800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:08.711838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:08.712566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:08.712849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:08.713122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:08.723583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:44:08.731134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:44:08.732341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:44:08.733502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 580/1149] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:44:08.739177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8047dd-4c59-4631-b2a9-a0bf35f3211e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:08.741378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:08.835985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8047dd-4c59-4631-b2a9-a0bf35f3211e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:08.869857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:08.870523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:08.870805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:08.871083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:08.871818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:08.872126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:08.873520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:08.881367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 570/1150] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:44:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:08.887469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:08.984506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28179c16-56f7-46e9-ab6f-87eed0cd4604 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:09.018362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:09.019238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:09.019521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:09.019794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] PATH_INFO: `/v3/users/20a6f28828784a77bf3c66a6fadea7ee` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:09.020984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] RBAC: Authorizing `identity:get_user(user_id=20a6f28828784a77bf3c66a6fadea7ee)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:09.022457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:09.024074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 581/1151] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:44:08 2022] GET /identity/v3/users/20a6f28828784a77bf3c66a6fadea7ee => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:09.029977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:09.090904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:09.128879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4685ac-ce17-47d3-aa25-99c352acbff3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:09.162611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70047133-91fa-46ca-a445-f9485a6cecb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:09.163468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70047133-91fa-46ca-a445-f9485a6cecb5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:09.163749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70047133-91fa-46ca-a445-f9485a6cecb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:09.164017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70047133-91fa-46ca-a445-f9485a6cecb5 None admin] PATH_INFO: `/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:09.165194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70047133-91fa-46ca-a445-f9485a6cecb5 None admin] RBAC: Authorizing `identity:get_project(project_id=1808f68ba63747f5b6e0cfa8d8d9447e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:09.166610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70047133-91fa-46ca-a445-f9485a6cecb5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:09.168283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 571/1152] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:44:09 2022] GET /identity/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:09.174448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:09.216494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:09.271470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ff9ac-af90-4adc-9a5a-5e3cb030bba4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:09.304155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a1165-13cd-4b3f-aa43-c382e4d19c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:09.305040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a1165-13cd-4b3f-aa43-c382e4d19c7c None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:09.305351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a1165-13cd-4b3f-aa43-c382e4d19c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:09.305625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a1165-13cd-4b3f-aa43-c382e4d19c7c None admin] PATH_INFO: `/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e/users/20a6f28828784a77bf3c66a6fadea7ee/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:09.307567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a1165-13cd-4b3f-aa43-c382e4d19c7c None admin] RBAC: Authorizing `identity:create_grant(project_id=1808f68ba63747f5b6e0cfa8d8d9447e, user_id=20a6f28828784a77bf3c66a6fadea7ee, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:09.309003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a1165-13cd-4b3f-aa43-c382e4d19c7c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:09.322059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 582/1153] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:44:09 2022] PUT /identity/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e/users/20a6f28828784a77bf3c66a6fadea7ee/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:11.490624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc4f5-02d9-4c74-b3d2-23c9fab624d6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:11.491145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc4f5-02d9-4c74-b3d2-23c9fab624d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:11.491145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc4f5-02d9-4c74-b3d2-23c9fab624d6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:11.492472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 572/1154] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:11.497664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66774928-5720-4e23-ac7d-d75fd851eaae None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:11.498111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66774928-5720-4e23-ac7d-d75fd851eaae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:11.498230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66774928-5720-4e23-ac7d-d75fd851eaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:11.515426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-66774928-5720-4e23-ac7d-d75fd851eaae None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:11.516614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-66774928-5720-4e23-ac7d-d75fd851eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:11.581826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 583/1155] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:11.605530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dc4f5-02d9-4c74-b3d2-23c9fab624d6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:11.710051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dc4f5-02d9-4c74-b3d2-23c9fab624d6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:11.739006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:11.744382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:11.745118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:11.745438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:11.745716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:11.756313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:44:11.764045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:44:11.765199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:44:11.766276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 573/1156] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:44:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:44:11.771454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66774928-5720-4e23-ac7d-d75fd851eaae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:11.802464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:11.870834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66774928-5720-4e23-ac7d-d75fd851eaae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:11.904054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:11.904722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:11.905003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:11.905317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:11.906057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:11.906361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:11.907878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:11.915853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 584/1157] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:44:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:11.922029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:12.020314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243262-e4b8-448b-a7c1-797230dca827 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:12.054421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:12.055270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:12.055545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:12.055812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] PATH_INFO: `/v3/users/20a6f28828784a77bf3c66a6fadea7ee` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:12.056969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] RBAC: Authorizing `identity:get_user(user_id=20a6f28828784a77bf3c66a6fadea7ee)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:12.058423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:12.060016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 574/1158] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:44:11 2022] GET /identity/v3/users/20a6f28828784a77bf3c66a6fadea7ee => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:12.066328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:12.157625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:12.164283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7af7a-d94b-4118-9dc7-a80ca9f90bd2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:12.197758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:12.198409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:12.198718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:12.198996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:12.263493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:12.264935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:12.353439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:12.361382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 585/1159] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:44:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:44:12.367257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:12.425391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:12.465388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa395b-d089-4fb6-b792-d126288d96d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:12.499141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea82b-4c58-434d-a166-35fa38a482ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:12.500006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea82b-4c58-434d-a166-35fa38a482ea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:12.500290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea82b-4c58-434d-a166-35fa38a482ea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:12.500568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea82b-4c58-434d-a166-35fa38a482ea None admin] PATH_INFO: `/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:12.501788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea82b-4c58-434d-a166-35fa38a482ea None admin] RBAC: Authorizing `identity:get_project(project_id=1808f68ba63747f5b6e0cfa8d8d9447e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:12.503209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea82b-4c58-434d-a166-35fa38a482ea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:12.504894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 575/1160] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:44:12 2022] GET /identity/v3/projects/1808f68ba63747f5b6e0cfa8d8d9447e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:12.511064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:12.607151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554840cc-d76e-46c1-94cd-0b6a9a037bde None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:12.640158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f3190-1b6f-4e50-bcbd-9ed0c041db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:12.640808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f3190-1b6f-4e50-bcbd-9ed0c041db44 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:12.641090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f3190-1b6f-4e50-bcbd-9ed0c041db44 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:12.641395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f3190-1b6f-4e50-bcbd-9ed0c041db44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:12.642550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f3190-1b6f-4e50-bcbd-9ed0c041db44 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=1808f68ba63747f5b6e0cfa8d8d9447e, user.id=20a6f28828784a77bf3c66a6fadea7ee) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:12.642929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f3190-1b6f-4e50-bcbd-9ed0c041db44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:12.644115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f3190-1b6f-4e50-bcbd-9ed0c041db44 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:12.651956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 586/1161] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:44:12 2022] GET /identity/v3/role_assignments?user.id=20a6f28828784a77bf3c66a6fadea7ee&scope.project.id=1808f68ba63747f5b6e0cfa8d8d9447e&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:14.839179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb818db9-6c15-494f-9bc2-3125a49d915b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:14.839747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb818db9-6c15-494f-9bc2-3125a49d915b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:14.839747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb818db9-6c15-494f-9bc2-3125a49d915b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:14.841069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 576/1162] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:14.846439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1650290-517c-41e6-9914-cedfe38493a8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:14.846860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1650290-517c-41e6-9914-cedfe38493a8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:14.847016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1650290-517c-41e6-9914-cedfe38493a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:14.863693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b1650290-517c-41e6-9914-cedfe38493a8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:14.864878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b1650290-517c-41e6-9914-cedfe38493a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:14.920765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:14.934842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 587/1163] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:14.958261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb818db9-6c15-494f-9bc2-3125a49d915b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:15.058631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb818db9-6c15-494f-9bc2-3125a49d915b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:15.085625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:15.093573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:15.094167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:15.094448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:15.094720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:15.164552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:15.166022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:15.266631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 577/1164] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:44:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:44:15.272848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1650290-517c-41e6-9914-cedfe38493a8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:15.369617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1650290-517c-41e6-9914-cedfe38493a8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:15.402002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e039e-24c4-422d-8659-6e870d79d1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:15.402799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e039e-24c4-422d-8659-6e870d79d1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:15.403076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e039e-24c4-422d-8659-6e870d79d1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:15.403349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e039e-24c4-422d-8659-6e870d79d1c5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:15.409883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e039e-24c4-422d-8659-6e870d79d1c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:15.411328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e039e-24c4-422d-8659-6e870d79d1c5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-951e039e-24c4-422d-8659-6e870d79d1c5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:44:15.425863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:44:15.427077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:44:15.428008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 588/1165] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:44:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:44:15.433114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:15.463882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:15.533854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae03a-9393-49a9-b8c9-19e9e90d5b44 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:15.566784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91214125-6934-4f84-a69f-a744712687e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:15.567440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214125-6934-4f84-a69f-a744712687e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:15.567718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214125-6934-4f84-a69f-a744712687e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:15.567988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214125-6934-4f84-a69f-a744712687e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:15.568698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214125-6934-4f84-a69f-a744712687e7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:15.568999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214125-6934-4f84-a69f-a744712687e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:15.570385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214125-6934-4f84-a69f-a744712687e7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:15.577294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 578/1166] 10.222.0.91 () {56 vars in 1115 bytes} [Thu May 26 13:44:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:17.763748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b0dd0-dc1e-46a8-9d1b-768a503715f4 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:17.764313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b0dd0-dc1e-46a8-9d1b-768a503715f4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:17.764313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b0dd0-dc1e-46a8-9d1b-768a503715f4 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:17.765680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 589/1167] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:17.771418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f66b8b-ca5c-4a39-9926-2660eff898ff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:17.771856 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f66b8b-ca5c-4a39-9926-2660eff898ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:17.771981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f66b8b-ca5c-4a39-9926-2660eff898ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:17.788456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4f66b8b-ca5c-4a39-9926-2660eff898ff None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:17.789677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4f66b8b-ca5c-4a39-9926-2660eff898ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:17.854822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 579/1168] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:17.877571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b0dd0-dc1e-46a8-9d1b-768a503715f4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:17.927375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:17.977806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b0dd0-dc1e-46a8-9d1b-768a503715f4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:18.009522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f98a3d-96fc-44c7-a95c-2eb85fa585f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:18.010171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98a3d-96fc-44c7-a95c-2eb85fa585f5 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:18.010445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98a3d-96fc-44c7-a95c-2eb85fa585f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:18.010717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98a3d-96fc-44c7-a95c-2eb85fa585f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:18.011452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f98a3d-96fc-44c7-a95c-2eb85fa585f5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:18.012814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f98a3d-96fc-44c7-a95c-2eb85fa585f5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:18.027120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 590/1169] 10.222.0.91 () {60 vars in 1138 bytes} [Thu May 26 13:44:17 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:20.270309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223249f3-8f64-4b04-bb51-a3a1ebe42384 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:20.270851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223249f3-8f64-4b04-bb51-a3a1ebe42384 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:20.271078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223249f3-8f64-4b04-bb51-a3a1ebe42384 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:20.273065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 580/1170] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:20.278573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b762-01d1-4870-9a4c-edcab5203f39 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:20.278982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b762-01d1-4870-9a4c-edcab5203f39 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:20.279130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b762-01d1-4870-9a4c-edcab5203f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:20.296716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2fb1b762-01d1-4870-9a4c-edcab5203f39 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:20.298504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2fb1b762-01d1-4870-9a4c-edcab5203f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:20.362862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 591/1171] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:20.386044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223249f3-8f64-4b04-bb51-a3a1ebe42384 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:20.410957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:20.487530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223249f3-8f64-4b04-bb51-a3a1ebe42384 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:20.520412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f830a-de6a-4bdb-9a2a-406ba8447b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:20.521290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f830a-de6a-4bdb-9a2a-406ba8447b34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:20.521576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f830a-de6a-4bdb-9a2a-406ba8447b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:20.521853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f830a-de6a-4bdb-9a2a-406ba8447b34 None admin] PATH_INFO: `/v3/domains/7ee3e188d8834820a751b5af264a6844` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:20.532889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f830a-de6a-4bdb-9a2a-406ba8447b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7ee3e188d8834820a751b5af264a6844)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:20.534383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f830a-de6a-4bdb-9a2a-406ba8447b34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:20.536034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 581/1172] 10.222.0.91 () {56 vars in 1183 bytes} [Thu May 26 13:44:20 2022] GET /identity/v3/domains/7ee3e188d8834820a751b5af264a6844 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:20.541883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1b762-01d1-4870-9a4c-edcab5203f39 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:20.580205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:20.638560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1b762-01d1-4870-9a4c-edcab5203f39 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:20.672679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b99dfa-b5b7-4f5c-bee6-9342a13d8faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:20.673373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b99dfa-b5b7-4f5c-bee6-9342a13d8faa None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:20.673661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b99dfa-b5b7-4f5c-bee6-9342a13d8faa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:20.673941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b99dfa-b5b7-4f5c-bee6-9342a13d8faa None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:20.674658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b99dfa-b5b7-4f5c-bee6-9342a13d8faa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:20.676089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b99dfa-b5b7-4f5c-bee6-9342a13d8faa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:20.699652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 592/1173] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:44:20 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:22.958988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e9db7-9c2b-45c4-a1b3-cd8627f7b8fa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:22.959568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e9db7-9c2b-45c4-a1b3-cd8627f7b8fa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:22.959568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e9db7-9c2b-45c4-a1b3-cd8627f7b8fa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:22.960946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 582/1174] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:44:22.968837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea1e8-9587-4d63-b531-198b62afc64d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:22.969303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea1e8-9587-4d63-b531-198b62afc64d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:22.969436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea1e8-9587-4d63-b531-198b62afc64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:22.988178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9caea1e8-9587-4d63-b531-198b62afc64d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:22.989413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9caea1e8-9587-4d63-b531-198b62afc64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:23.056513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 593/1175] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:23.081050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e9db7-9c2b-45c4-a1b3-cd8627f7b8fa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:23.177706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:23.198770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e9db7-9c2b-45c4-a1b3-cd8627f7b8fa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:23.233333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482a0d8-00e2-448c-a35a-441b808a2bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:23.234113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a0d8-00e2-448c-a35a-441b808a2bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:23.234392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a0d8-00e2-448c-a35a-441b808a2bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:23.234662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a0d8-00e2-448c-a35a-441b808a2bb8 None admin] PATH_INFO: `/v3/domains/7ee3e188d8834820a751b5af264a6844` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:23.236493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482a0d8-00e2-448c-a35a-441b808a2bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7ee3e188d8834820a751b5af264a6844)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:23.237963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482a0d8-00e2-448c-a35a-441b808a2bb8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:23.239633 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 583/1176] 10.222.0.91 () {56 vars in 1183 bytes} [Thu May 26 13:44:23 2022] GET /identity/v3/domains/7ee3e188d8834820a751b5af264a6844 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:23.246275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caea1e8-9587-4d63-b531-198b62afc64d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:23.268939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:23.359716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caea1e8-9587-4d63-b531-198b62afc64d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:23.395109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850bcfe-5ade-4874-9ce6-4051b69b572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:23.395783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850bcfe-5ade-4874-9ce6-4051b69b572d None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:23.396063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850bcfe-5ade-4874-9ce6-4051b69b572d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:23.396337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850bcfe-5ade-4874-9ce6-4051b69b572d None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:23.397056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850bcfe-5ade-4874-9ce6-4051b69b572d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:23.398536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850bcfe-5ade-4874-9ce6-4051b69b572d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:23.419982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 594/1177] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:44:23 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:44:25.705548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ad39-c644-4d09-9bcb-0277c848821e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:25.706094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ad39-c644-4d09-9bcb-0277c848821e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:25.706201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ad39-c644-4d09-9bcb-0277c848821e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:25.707486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 584/1178] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:25.712616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72eb61-904a-4581-ac78-6bf07e117845 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:25.713044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72eb61-904a-4581-ac78-6bf07e117845 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:25.713205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72eb61-904a-4581-ac78-6bf07e117845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:25.730309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7f72eb61-904a-4581-ac78-6bf07e117845 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:25.731495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7f72eb61-904a-4581-ac78-6bf07e117845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:25.796201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 595/1179] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:25.819120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ad39-c644-4d09-9bcb-0277c848821e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:25.918687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ad39-c644-4d09-9bcb-0277c848821e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:25.951980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:25.952757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:25.953038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:25.953344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:25.964216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:44:25.972370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:44:25.973550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:44:25.974618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 585/1180] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:44:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:44:25.979700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72eb61-904a-4581-ac78-6bf07e117845 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:25.988024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:26.077594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72eb61-904a-4581-ac78-6bf07e117845 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:26.112235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:26.112894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:26.113197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:26.113476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:26.114215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:26.114520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:26.115883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:26.123607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 596/1181] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:44:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:26.129353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:26.141334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:26.227240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b532275-d903-4408-b0ea-35dcfca6cd75 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:26.260107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:26.260963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:26.261275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:26.261550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] PATH_INFO: `/v3/users/d427fe6128224655babec52b039845b3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:26.274035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] RBAC: Authorizing `identity:get_user(user_id=d427fe6128224655babec52b039845b3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:26.275461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:26.277081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 586/1182] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:44:26 2022] GET /identity/v3/users/d427fe6128224655babec52b039845b3 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:26.283266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:26.347556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:26.379017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39aea1-53e2-4568-b97e-da2ed587aa5d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:26.417766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:26.418421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:26.418698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:26.418968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:26.484722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:26.486131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:26.548069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:26.581272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 597/1183] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:44:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:44:26.587088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:26.651418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:26.684185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691ae54-f601-4e3f-9f2e-b98e669f86b0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:26.721365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee51fe-b3f9-4af5-8be8-3c8fe867f157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:26.722649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee51fe-b3f9-4af5-8be8-3c8fe867f157 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:26.723071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee51fe-b3f9-4af5-8be8-3c8fe867f157 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:26.723481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee51fe-b3f9-4af5-8be8-3c8fe867f157 None admin] PATH_INFO: `/v3/projects/8bd1c567972f4fb891550050e22204d9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:26.725227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee51fe-b3f9-4af5-8be8-3c8fe867f157 None admin] RBAC: Authorizing `identity:get_project(project_id=8bd1c567972f4fb891550050e22204d9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:26.727429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee51fe-b3f9-4af5-8be8-3c8fe867f157 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:26.730036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 587/1184] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:44:26 2022] GET /identity/v3/projects/8bd1c567972f4fb891550050e22204d9 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:26.736162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:26.837807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b7c12-48d7-4ecd-9a64-f36f007254c7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:26.869877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05aca8-9593-4522-9851-9fd8ec16be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:26.870525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05aca8-9593-4522-9851-9fd8ec16be38 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:26.870810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05aca8-9593-4522-9851-9fd8ec16be38 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:26.871086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05aca8-9593-4522-9851-9fd8ec16be38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:26.871905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05aca8-9593-4522-9851-9fd8ec16be38 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=8bd1c567972f4fb891550050e22204d9, user.id=d427fe6128224655babec52b039845b3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:26.872216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05aca8-9593-4522-9851-9fd8ec16be38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:26.873673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05aca8-9593-4522-9851-9fd8ec16be38 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:26.879463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 598/1185] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:44:26 2022] GET /identity/v3/role_assignments?user.id=d427fe6128224655babec52b039845b3&scope.project.id=8bd1c567972f4fb891550050e22204d9&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:29.042527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956592-d1b9-4bc9-8fec-a0c00756ad8b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:29.043068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956592-d1b9-4bc9-8fec-a0c00756ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:29.043068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956592-d1b9-4bc9-8fec-a0c00756ad8b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:29.044439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 588/1186] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:29.050460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21fe4a-a9f7-45f7-97c9-4c3b1af71f6a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:29.050850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21fe4a-a9f7-45f7-97c9-4c3b1af71f6a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:29.051011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21fe4a-a9f7-45f7-97c9-4c3b1af71f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:29.067726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9c21fe4a-a9f7-45f7-97c9-4c3b1af71f6a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:29.068908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c21fe4a-a9f7-45f7-97c9-4c3b1af71f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:29.093989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:29.134660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 599/1187] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:29.156483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df956592-d1b9-4bc9-8fec-a0c00756ad8b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:29.256689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df956592-d1b9-4bc9-8fec-a0c00756ad8b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:29.288939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:29.289808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:29.290086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:29.290356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:29.292956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:29.301915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:44:29.309497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:44:29.310687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:44:29.311890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 589/1188] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:44:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:44:29.316673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21fe4a-a9f7-45f7-97c9-4c3b1af71f6a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:29.413044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21fe4a-a9f7-45f7-97c9-4c3b1af71f6a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:29.445951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:29.446608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:29.446892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:29.447170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:29.447912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:29.448220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:29.449630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:29.457987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 600/1189] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:44:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:29.463796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:29.561018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd922feb-bb73-4eba-b17f-468c6709440b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:29.593247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:29.594098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:29.594988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:29.595267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] PATH_INFO: `/v3/users/d427fe6128224655babec52b039845b3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:29.596446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] RBAC: Authorizing `identity:get_user(user_id=d427fe6128224655babec52b039845b3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:29.597924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:29.599526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 590/1190] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:44:29 2022] GET /identity/v3/users/d427fe6128224655babec52b039845b3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:29.605039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:29.631994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:29.704664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa828d-3b71-477d-b198-1f78472e3b13 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:29.737625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5bfa6-a9fc-4f55-8087-32d8686ee877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:29.738476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5bfa6-a9fc-4f55-8087-32d8686ee877 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:29.738761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5bfa6-a9fc-4f55-8087-32d8686ee877 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:29.739040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5bfa6-a9fc-4f55-8087-32d8686ee877 None admin] PATH_INFO: `/v3/projects/8bd1c567972f4fb891550050e22204d9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:29.740208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5bfa6-a9fc-4f55-8087-32d8686ee877 None admin] RBAC: Authorizing `identity:get_project(project_id=8bd1c567972f4fb891550050e22204d9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:29.741673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5bfa6-a9fc-4f55-8087-32d8686ee877 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:29.743340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 601/1191] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:44:29 2022] GET /identity/v3/projects/8bd1c567972f4fb891550050e22204d9 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:29.749267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:29.809812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:29.847085 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af514c6f-3dd4-45b5-bd06-0fcf34b98f01 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:29.879289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a4793-6f20-48e8-b47d-21cb58a818bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:29.880171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4793-6f20-48e8-b47d-21cb58a818bf None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:29.880456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4793-6f20-48e8-b47d-21cb58a818bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:29.880728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4793-6f20-48e8-b47d-21cb58a818bf None admin] PATH_INFO: `/v3/projects/8bd1c567972f4fb891550050e22204d9/users/d427fe6128224655babec52b039845b3/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:29.882691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4793-6f20-48e8-b47d-21cb58a818bf None admin] RBAC: Authorizing `identity:create_grant(project_id=8bd1c567972f4fb891550050e22204d9, user_id=d427fe6128224655babec52b039845b3, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:29.884109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4793-6f20-48e8-b47d-21cb58a818bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:29.896350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 591/1192] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:44:29 2022] PUT /identity/v3/projects/8bd1c567972f4fb891550050e22204d9/users/d427fe6128224655babec52b039845b3/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:32.053809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b798e0-3126-4b58-95f8-be914d9160b7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:32.054369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b798e0-3126-4b58-95f8-be914d9160b7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:32.054369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b798e0-3126-4b58-95f8-be914d9160b7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:32.055739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 602/1193] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:32.060892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecc85f-7372-49e8-80c4-68badf0a1955 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:32.061322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecc85f-7372-49e8-80c4-68badf0a1955 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:32.061487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecc85f-7372-49e8-80c4-68badf0a1955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:32.078305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-baecc85f-7372-49e8-80c4-68badf0a1955 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:32.080028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-baecc85f-7372-49e8-80c4-68badf0a1955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:32.147222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 592/1194] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:44:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:44:32.170551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b798e0-3126-4b58-95f8-be914d9160b7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:32.260540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:32.270807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b798e0-3126-4b58-95f8-be914d9160b7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:32.302933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:32.303719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:32.304052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:32.304277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:32.314621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:44:32.322120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:44:32.323236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:44:32.324134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 603/1195] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:44:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:44:32.329928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecc85f-7372-49e8-80c4-68badf0a1955 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:32.382584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:32.429980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecc85f-7372-49e8-80c4-68badf0a1955 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:32.463763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:32.464430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:32.464714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:32.464987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:32.465751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:32.466058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:32.467419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:32.475330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 593/1196] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:44:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:32.481085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:32.577703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded546c-6fde-4f08-bc73-2424c79f041b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:32.609288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:32.610657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:32.610934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:32.611202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] PATH_INFO: `/v3/users/d427fe6128224655babec52b039845b3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:32.612370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] RBAC: Authorizing `identity:get_user(user_id=d427fe6128224655babec52b039845b3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:32.613837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:32.615427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 604/1197] 10.222.0.91 () {56 vars in 1177 bytes} [Thu May 26 13:44:32 2022] GET /identity/v3/users/d427fe6128224655babec52b039845b3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:32.621096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:32.718366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb2c9f-d563-4247-9b21-0bca045eaf04 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:32.734276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:32.753083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:32.753777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:32.754064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:32.754340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:32.823415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:32.824869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:32.921436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 594/1198] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:44:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:44:32.927799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:32.949675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:33.025471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2415a7-8db4-42db-b5d8-5b81e9328737 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:33.059239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d63f34-5a90-433d-9ce0-ed3d733dc0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:33.060102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63f34-5a90-433d-9ce0-ed3d733dc0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:33.060386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63f34-5a90-433d-9ce0-ed3d733dc0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:33.060662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63f34-5a90-433d-9ce0-ed3d733dc0d8 None admin] PATH_INFO: `/v3/projects/8bd1c567972f4fb891550050e22204d9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:33.061846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d63f34-5a90-433d-9ce0-ed3d733dc0d8 None admin] RBAC: Authorizing `identity:get_project(project_id=8bd1c567972f4fb891550050e22204d9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:33.063314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d63f34-5a90-433d-9ce0-ed3d733dc0d8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:33.064998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 605/1199] 10.222.0.91 () {56 vars in 1186 bytes} [Thu May 26 13:44:32 2022] GET /identity/v3/projects/8bd1c567972f4fb891550050e22204d9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:33.071054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:33.088325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:33.167898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a84d7-7e0c-47f4-bba2-57238958d16d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:33.200888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042a1cb-f812-4471-812a-46e35d48f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:33.201565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042a1cb-f812-4471-812a-46e35d48f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:33.201849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042a1cb-f812-4471-812a-46e35d48f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:33.202125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042a1cb-f812-4471-812a-46e35d48f5f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:33.202951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042a1cb-f812-4471-812a-46e35d48f5f5 None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=8bd1c567972f4fb891550050e22204d9, user.id=d427fe6128224655babec52b039845b3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:33.203261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042a1cb-f812-4471-812a-46e35d48f5f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:33.205211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042a1cb-f812-4471-812a-46e35d48f5f5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:33.211807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 595/1200] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:44:33 2022] GET /identity/v3/role_assignments?user.id=d427fe6128224655babec52b039845b3&scope.project.id=8bd1c567972f4fb891550050e22204d9&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:43.055007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=56 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:43.209792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=60 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:34.547904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831f4d-1403-42c9-b602-bea43bfc0afa None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:34.548871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831f4d-1403-42c9-b602-bea43bfc0afa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:34.548871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831f4d-1403-42c9-b602-bea43bfc0afa None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.549852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 606/1201] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:34.561690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27770-94ca-4609-921e-212f67d93e0a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:34.562534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27770-94ca-4609-921e-212f67d93e0a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:34.562701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27770-94ca-4609-921e-212f67d93e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.592745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ce27770-94ca-4609-921e-212f67d93e0a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:34.595156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ce27770-94ca-4609-921e-212f67d93e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:34.721417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 596/1202] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:45:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:45:34.748761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06831f4d-1403-42c9-b602-bea43bfc0afa None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:34.902596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06831f4d-1403-42c9-b602-bea43bfc0afa None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:34.958037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:34.958819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:34.959142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:34.959375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.971895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:34.973408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:34.992438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:34.994373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:34.995837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:34.995837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:34.995837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:34.995837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:34.995837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 607/1203] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:35.003760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce27770-94ca-4609-921e-212f67d93e0a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:35.150056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce27770-94ca-4609-921e-212f67d93e0a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:35.198529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:35.207566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73701b-502a-41ed-9f2b-b9b648b9c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:35.208409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73701b-502a-41ed-9f2b-b9b648b9c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:35.208699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73701b-502a-41ed-9f2b-b9b648b9c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:35.208973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73701b-502a-41ed-9f2b-b9b648b9c1a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:35.209866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73701b-502a-41ed-9f2b-b9b648b9c1a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:35.210283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73701b-502a-41ed-9f2b-b9b648b9c1a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:35.211697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73701b-502a-41ed-9f2b-b9b648b9c1a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:35.227303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 597/1204] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:35.232860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:35.304858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:35.374654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10db1-2d30-4d9d-9b03-37d0eaa72f67 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:35.417561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41748d-1fbd-4ce8-873e-e6ed7eac4d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:35.418959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41748d-1fbd-4ce8-873e-e6ed7eac4d32 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:35.419454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41748d-1fbd-4ce8-873e-e6ed7eac4d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:35.419974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41748d-1fbd-4ce8-873e-e6ed7eac4d32 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:35.421364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41748d-1fbd-4ce8-873e-e6ed7eac4d32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:35.422979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41748d-1fbd-4ce8-873e-e6ed7eac4d32 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:35.452148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 608/1205] 10.222.0.91 () {60 vars in 1132 bytes} [Thu May 26 13:45:35 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:38.394129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550e646-ccdc-40e8-86d0-abc8e2a1086c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.394711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550e646-ccdc-40e8-86d0-abc8e2a1086c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.394711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550e646-ccdc-40e8-86d0-abc8e2a1086c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.397490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 598/1206] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:38.403527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c705-3c97-4aa3-bf75-400a4af13872 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.404136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c705-3c97-4aa3-bf75-400a4af13872 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.404136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c705-3c97-4aa3-bf75-400a4af13872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.428428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b7d0c705-3c97-4aa3-bf75-400a4af13872 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:38.429822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b7d0c705-3c97-4aa3-bf75-400a4af13872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:38.519499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 609/1207] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:45:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:45:38.552189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550e646-ccdc-40e8-86d0-abc8e2a1086c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.700387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550e646-ccdc-40e8-86d0-abc8e2a1086c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.752798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.753664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.754121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.754401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.771389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:45:38.781921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:38.782983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:38.783918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 599/1208] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:38.789373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c705-3c97-4aa3-bf75-400a4af13872 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.851749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:38.945882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c705-3c97-4aa3-bf75-400a4af13872 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.993336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.997086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.997086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.997086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.997086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:38.997086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.997496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.013261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 610/1209] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:45:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:45:39.015295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:39.087221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:39.147591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1c3a9-aa1e-46aa-99d2-da99831b23da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:39.191933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.192922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.193349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.193725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.207939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.209621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:39.232478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:39.234688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:39.235868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:39.235868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:39.235868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:39.235868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:39.237336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 600/1210] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:39.241034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:39.383361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:39.387260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074de8d-0a03-4010-91ae-767c05ed1c15 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:39.434801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.435858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.435858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.436134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.437399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:39.437921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.439069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.452151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 611/1211] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:39.457442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:39.578312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92188f99-3684-4799-b014-5d16431f3c29 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:39.579872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:39.622886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.623782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.624635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.624930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.633705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.635304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:45:39.652119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:39.653198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:39.658431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 601/1212] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:39.659620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:39.791354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0091e2-4d2f-4bb5-8c1b-3d984365fd03 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:39.841993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.841993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.842274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.842445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.844007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:39.844551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.846177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.866851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 612/1213] 10.222.0.91 () {56 vars in 1143 bytes} [Thu May 26 13:45:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:39.872523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:40.003521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7defd-2a26-43ef-99a6-98f58388d98a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:40.045553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:40.046732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:40.047017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:40.047316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:40.080666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:40.129202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:40.131017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:40.271869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 602/1214] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:45:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:45:40.277475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:40.304055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:40.408409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b65f3-a75c-45e8-bd14-0b877d2b846f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:40.450867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:40.451851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:40.452159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:40.452520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:40.466410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:40.468725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:40.496247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:40.497427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:40.498438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:40.498438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:40.498438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:40.498438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:40.502463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 613/1215] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:45:40.503514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:40.589638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:40.636997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602e030-d3f3-45e2-b070-8fb7a0b9b959 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:40.684850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d50332-d641-412c-abc3-c944566980bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:40.685840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50332-d641-412c-abc3-c944566980bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:40.686126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50332-d641-412c-abc3-c944566980bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:40.686466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50332-d641-412c-abc3-c944566980bf None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:40.687403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d50332-d641-412c-abc3-c944566980bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:40.687728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d50332-d641-412c-abc3-c944566980bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:40.689386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d50332-d641-412c-abc3-c944566980bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:40.705321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 603/1216] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:40.711374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:40.801460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:40.843268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615fdec-4492-4b86-b951-af9a32d982c8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:40.884085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:40.884900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:40.885201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:40.885489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:40.893308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:40.894780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:45:40.918814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:40.919998 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:40.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:40.923588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 614/1217] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:45:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:45:40.932279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d50332-d641-412c-abc3-c944566980bf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.078608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d50332-d641-412c-abc3-c944566980bf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.102764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:41.118774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba9da4-c908-432c-9ceb-7540ff97075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.119467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9da4-c908-432c-9ceb-7540ff97075e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.119748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9da4-c908-432c-9ceb-7540ff97075e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.120021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9da4-c908-432c-9ceb-7540ff97075e None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.120866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba9da4-c908-432c-9ceb-7540ff97075e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:41.121198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba9da4-c908-432c-9ceb-7540ff97075e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.122691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba9da4-c908-432c-9ceb-7540ff97075e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.137471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 604/1218] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:45:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:45:41.144929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.272277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455a8e-d784-448c-97fb-d74c96ce3b1c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.306901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:41.315441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52ae14-2991-4238-892a-92896b71a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.316310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52ae14-2991-4238-892a-92896b71a78d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.316603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52ae14-2991-4238-892a-92896b71a78d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.316898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52ae14-2991-4238-892a-92896b71a78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.317963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52ae14-2991-4238-892a-92896b71a78d None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=997348872c3241f4a14c6b2fe9b66211) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:41.318311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52ae14-2991-4238-892a-92896b71a78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.320068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52ae14-2991-4238-892a-92896b71a78d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.326960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 615/1219] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:45:41 2022] GET /identity/v3/role_assignments?user.id=997348872c3241f4a14c6b2fe9b66211&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:43.665940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708b372-b12f-48f7-88fa-499d4bff6b76 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:43.666718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708b372-b12f-48f7-88fa-499d4bff6b76 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:43.666718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708b372-b12f-48f7-88fa-499d4bff6b76 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:43.667715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 605/1220] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:43.676875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2f7b4-b8ec-4e61-920e-ab9b4c44d95b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:43.676875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2f7b4-b8ec-4e61-920e-ab9b4c44d95b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:43.676875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2f7b4-b8ec-4e61-920e-ab9b4c44d95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:43.697292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-48c2f7b4-b8ec-4e61-920e-ab9b4c44d95b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:43.698740 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-48c2f7b4-b8ec-4e61-920e-ab9b4c44d95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:43.788371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 616/1221] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:45:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:45:43.815363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708b372-b12f-48f7-88fa-499d4bff6b76 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:43.948442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708b372-b12f-48f7-88fa-499d4bff6b76 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:43.995868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:43.996706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:43.997142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:43.997441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:44.008906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:45:44.016905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:44.018541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:44.020157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 606/1222] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:45:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:44.024629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2f7b4-b8ec-4e61-920e-ab9b4c44d95b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:44.136890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2f7b4-b8ec-4e61-920e-ab9b4c44d95b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:44.164374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:44.180135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:44.180815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:44.181093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:44.181393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:44.182144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:44.183693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:44.188305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:44.198995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 617/1223] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:45:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:45:44.205208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:44.256174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:44.336340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af5cd7-3e7f-4a97-8e12-acebb5d9ff9a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:44.381591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:44.382398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:44.382812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:44.383116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:44.398231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:44.399767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:44.418846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:44.420025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:44.426231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:44.426231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:44.426231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:44.426231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:44.426231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 607/1224] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:44.426665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:44.541642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0b32c-4c55-4479-9414-00a5c766a799 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:44.583790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:44.584884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:44.585331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:44.585737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:44.586592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:44.587027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:44.588534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:44.601771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 618/1225] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:44.608262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:44.718703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:44.726533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46321d-a38a-469b-924c-ff9eac424186 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:44.765897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:44.766746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:44.766984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:44.767256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:44.775308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:44.776999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:45:44.791619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:44.793228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:44.794512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 608/1226] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:44.801316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:44.871347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:44.938055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74b08a-a709-42ee-aba9-553a74ade76d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:44.987508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:44.988262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:44.988553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:44.988789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:44.989909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:44.990214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:44.991508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:45.016955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 619/1227] 10.222.0.91 () {56 vars in 1143 bytes} [Thu May 26 13:45:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:45.023919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:45.143996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ee83e-d82c-498b-8f76-9139d7258bbd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:45.189207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:45.190116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:45.190455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:45.190822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:45.203654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:45.205107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:45.223477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:45.224644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:45.226041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:45.226041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:45.226041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:45.226041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:45.226041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 609/1228] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:45.231151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:45.344481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d5547-3d97-42dc-967e-f38e0d3ff83a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:45.346949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:45.388684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:45.390687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:45.390687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:45.390906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:45.391990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:45.392316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:45.394331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:45.409694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 620/1229] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:45.415591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:45.427380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:45.537686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0f578-c47d-40f6-b5c3-711a1cedec83 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:45.588358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:45.589169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:45.589484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:45.589746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:45.598364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:45.599814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:45:45.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:45.616851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:45.618116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 610/1230] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:45:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:45.622151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:45.740424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c991-a574-4e27-b668-26b203f3cddf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:45.779039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e19cf7-1ec3-4356-9f1a-6dfcdac96f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:45.780331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e19cf7-1ec3-4356-9f1a-6dfcdac96f4c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:45.780731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e19cf7-1ec3-4356-9f1a-6dfcdac96f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:45.781613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e19cf7-1ec3-4356-9f1a-6dfcdac96f4c None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:45.783028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e19cf7-1ec3-4356-9f1a-6dfcdac96f4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:45.783613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e19cf7-1ec3-4356-9f1a-6dfcdac96f4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:45.786099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e19cf7-1ec3-4356-9f1a-6dfcdac96f4c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:45.801372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 621/1231] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:45:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:45.807461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:45.888310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:45.932216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c457fb7-2695-4462-ad10-11689845f896 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:45.983743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac0e5d-3668-40c8-a3f8-42a1c10bf7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:45.984839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0e5d-3668-40c8-a3f8-42a1c10bf7d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:45.985337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0e5d-3668-40c8-a3f8-42a1c10bf7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:45.985719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0e5d-3668-40c8-a3f8-42a1c10bf7d7 None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/997348872c3241f4a14c6b2fe9b66211/roles/f423841743584b218cefb221f7b3edff` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:46.004965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac0e5d-3668-40c8-a3f8-42a1c10bf7d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=997348872c3241f4a14c6b2fe9b66211, role_id=f423841743584b218cefb221f7b3edff)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:46.006782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac0e5d-3668-40c8-a3f8-42a1c10bf7d7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:46.024661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 611/1232] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:45:45 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/997348872c3241f4a14c6b2fe9b66211/roles/f423841743584b218cefb221f7b3edff => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:48.245688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9244644-a06d-47a7-87a1-2ecf83703363 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:48.246215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9244644-a06d-47a7-87a1-2ecf83703363 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:48.246333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9244644-a06d-47a7-87a1-2ecf83703363 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:48.247654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 622/1233] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:48.255026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac7600-2ff6-4ebe-8fed-cde54345ed81 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:48.255026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac7600-2ff6-4ebe-8fed-cde54345ed81 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:48.256224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac7600-2ff6-4ebe-8fed-cde54345ed81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:48.279259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-70ac7600-2ff6-4ebe-8fed-cde54345ed81 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:48.280534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-70ac7600-2ff6-4ebe-8fed-cde54345ed81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:48.374733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 612/1234] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:45:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:45:48.399963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9244644-a06d-47a7-87a1-2ecf83703363 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:48.432340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:48.529015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9244644-a06d-47a7-87a1-2ecf83703363 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:48.571715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:48.572531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:48.572840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:48.573115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:48.586304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:45:48.595287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:48.597370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:48.599119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 623/1235] 10.222.0.91 () {56 vars in 1102 bytes} [Thu May 26 13:45:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:48.602729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac7600-2ff6-4ebe-8fed-cde54345ed81 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:48.673189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:48.739704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac7600-2ff6-4ebe-8fed-cde54345ed81 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:48.780648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:48.781613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:48.782030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:48.782432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:48.783314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:48.783624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:48.785269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:48.796169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 613/1236] 10.222.0.91 () {56 vars in 1103 bytes} [Thu May 26 13:45:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:45:48.802670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:48.902653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:48.922304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9df34e-4472-4258-999d-451e3ec8dfda None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:48.973379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:48.974448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:48.974889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:48.975236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:48.989117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:48.991142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:49.013090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:49.014713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:49.015744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:49.015744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:49.015744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:49.015744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:49.015744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 624/1237] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:49.021382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:49.143447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97242be9-02bc-41c3-b298-5bac8d6c5a40 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:49.156245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:49.185703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:49.186973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:49.187297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:49.187576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:49.188429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:49.188765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:49.190300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:49.204052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 614/1238] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:49.210361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:49.331749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e0ae3-6eb3-4a7b-89f6-5336cff5b594 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:49.370781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:49.371895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:49.372175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:49.372448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:49.376777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:49.383247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:49.384893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:45:49.401128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:49.402390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:49.403557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 625/1239] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:49.408175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:49.523306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c40ce-6692-4b0e-884c-93eed8b51f33 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:49.563180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:49.564182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:49.564508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:49.564740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:49.565612 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:49.565920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:49.567358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:49.587931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 615/1240] 10.222.0.91 () {56 vars in 1143 bytes} [Thu May 26 13:45:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:49.596211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:49.718942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861353a-23a2-4289-8fd7-e4a48bcb54ca None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:49.768793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:49.769540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:49.769818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:49.770068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:49.847537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:49.856172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:49.857754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.005268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 626/1241] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:45:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:45:50.007491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.045620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:50.140836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253b7a4-2af6-468f-a4cf-0b2828c89375 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.182390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.183192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.183477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.183757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.197695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:50.200312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:50.221916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:50.223215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:50.224331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:50.224331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:50.224331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:50.224331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:50.224331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 616/1242] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:50.230126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.359623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:50.365801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b34409-2d1b-4520-8cd2-4d88309d79c8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.405086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.405845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.406240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.406644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.407580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:50.407894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:50.410037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.428549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 627/1243] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:50.430262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.526988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:50.553705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71609860-4dca-4f70-aaa3-fee1c4359739 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.600443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.601567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.602024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.602492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.611638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:50.613230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:45:50.629876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:50.631432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:50.632709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 617/1244] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:45:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:50.639609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.774562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156a3f1-9dfb-450d-9f16-0140c0a0a8da None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.811940 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466921c-4441-4613-b57c-3d7745964d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.812610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466921c-4441-4613-b57c-3d7745964d06 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.812907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466921c-4441-4613-b57c-3d7745964d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.813195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466921c-4441-4613-b57c-3d7745964d06 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.813990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466921c-4441-4613-b57c-3d7745964d06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:50.814934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466921c-4441-4613-b57c-3d7745964d06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:50.815742 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466921c-4441-4613-b57c-3d7745964d06 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.828313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 628/1245] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:45:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:50.834254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.952757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f5bc-8929-4a9e-b0f2-9c9a54eaf615 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.996676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363631c-a413-4f4f-a281-00c7c87e890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.997455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363631c-a413-4f4f-a281-00c7c87e890c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.997733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363631c-a413-4f4f-a281-00c7c87e890c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.998005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363631c-a413-4f4f-a281-00c7c87e890c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.999428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363631c-a413-4f4f-a281-00c7c87e890c None admin] RBAC: Adding query filter params (role.id=f423841743584b218cefb221f7b3edff, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=997348872c3241f4a14c6b2fe9b66211) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:50.999778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363631c-a413-4f4f-a281-00c7c87e890c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.001189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363631c-a413-4f4f-a281-00c7c87e890c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.008179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 618/1246] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:45:50 2022] GET /identity/v3/role_assignments?user.id=997348872c3241f4a14c6b2fe9b66211&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=f423841743584b218cefb221f7b3edff => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:53.346507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec477-35f0-4534-a89b-a033aacfae4d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:53.347178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec477-35f0-4534-a89b-a033aacfae4d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:53.347178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec477-35f0-4534-a89b-a033aacfae4d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:53.348337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 629/1247] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:53.353620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a322e-b54c-42dd-9c7e-df906efada3c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:53.354132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a322e-b54c-42dd-9c7e-df906efada3c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:53.354245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a322e-b54c-42dd-9c7e-df906efada3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:53.360106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:53.375640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-447a322e-b54c-42dd-9c7e-df906efada3c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:53.376894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-447a322e-b54c-42dd-9c7e-df906efada3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:53.456304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 619/1248] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:45:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:45:53.488652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deec477-35f0-4534-a89b-a033aacfae4d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:53.510198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:53.600649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deec477-35f0-4534-a89b-a033aacfae4d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:53.640524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:53.641545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:53.641866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:53.642161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:53.653679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:45:53.665666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:53.667426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:53.668916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 630/1249] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:45:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:45:53.673949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a322e-b54c-42dd-9c7e-df906efada3c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:53.791176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a322e-b54c-42dd-9c7e-df906efada3c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:53.829772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:53.835095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:53.835957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:53.836230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:53.836509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:53.837507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:53.837877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:53.840099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:53.851284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 620/1250] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:45:53.857199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:53.955447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:53.989292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca082a8-a63e-4b98-a7ce-a905d2d87c2f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:54.037214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:54.037622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:54.037982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:54.038359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:54.052240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:54.054004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:54.074371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:54.076230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:54.078600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:54.078600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:54.078600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:54.078600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:54.078600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 631/1251] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:54.082328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:54.216891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e05920-acd3-4440-9b8f-f44355a5ecf3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:54.256087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:54.256832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:54.257141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:54.257534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:54.258435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:54.258786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:54.260244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:54.276399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 621/1252] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:54.283714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:54.416356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea58cf-de4e-4379-b8a0-2614d403f480 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:54.451039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:54.460881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:54.461729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:54.462015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:54.462322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:54.473136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:54.475032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:45:54.492131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:54.493555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:54.494518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 632/1253] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:54.500099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:54.571044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:54.625586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4787d9-4205-4a8c-85ec-032c54121c0d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:54.667430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:54.668323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:54.668729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:54.669074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:54.669997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:54.670343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:54.671796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:54.692818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 622/1254] 10.222.0.91 () {56 vars in 1143 bytes} [Thu May 26 13:45:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:54.699425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:54.815451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699cca0-7ca6-4ed3-af4b-3ddf21e5c0fc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:54.855826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:54.856499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:54.856787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:54.857096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:54.921643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:54.937002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:54.938527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:55.069012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 633/1255] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:45:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:45:55.075327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:55.184758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5dd0d-5a5a-4dd6-9038-d0cc87165ea6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:55.202034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:55.224548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:55.225389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:55.225669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:55.225939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:55.239230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:55.240712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:55.259613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:55.260696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:55.261930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:55.261930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:55.261930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:55.261930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:55.261930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 623/1256] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:45:55.266787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:55.364546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:55.379308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae454bf7-e588-491c-ac8d-0a12f2ba4361 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:55.426924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:55.427587 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:55.427907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:55.428454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:55.430306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:55.430306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:55.430598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:55.442550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 634/1257] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:55.448412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:55.560837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f45f2-f9ba-4352-85b5-d0eab6b5e90a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:55.594780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193215a-35ed-4943-818e-35ace907764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:55.595587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193215a-35ed-4943-818e-35ace907764c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:55.595870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193215a-35ed-4943-818e-35ace907764c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:55.596146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193215a-35ed-4943-818e-35ace907764c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:55.602805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193215a-35ed-4943-818e-35ace907764c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:55.604248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193215a-35ed-4943-818e-35ace907764c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8193215a-35ed-4943-818e-35ace907764c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:45:55.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:55.620621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:45:55.621796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 624/1258] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:45:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:55.627347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:55.748367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269fd67-1d5e-47f1-a4bf-4e0e5ddc5f2e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:55.789236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb6529-4fbf-4305-b0af-5f6a2efdc42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:55.789913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb6529-4fbf-4305-b0af-5f6a2efdc42a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:55.790234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb6529-4fbf-4305-b0af-5f6a2efdc42a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:55.790512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb6529-4fbf-4305-b0af-5f6a2efdc42a None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:55.791309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb6529-4fbf-4305-b0af-5f6a2efdc42a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:55.791622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb6529-4fbf-4305-b0af-5f6a2efdc42a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:55.793340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb6529-4fbf-4305-b0af-5f6a2efdc42a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:55.807008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 635/1259] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:45:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:55.813889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193215a-35ed-4943-818e-35ace907764c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:55.839458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:55.912778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193215a-35ed-4943-818e-35ace907764c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:55.948129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec055f-58c2-4cf8-84b2-9f5fc48af35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:55.949012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec055f-58c2-4cf8-84b2-9f5fc48af35a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:55.949383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec055f-58c2-4cf8-84b2-9f5fc48af35a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:55.949707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec055f-58c2-4cf8-84b2-9f5fc48af35a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:55.950753 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec055f-58c2-4cf8-84b2-9f5fc48af35a None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=997348872c3241f4a14c6b2fe9b66211) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:55.951092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec055f-58c2-4cf8-84b2-9f5fc48af35a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:55.952860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec055f-58c2-4cf8-84b2-9f5fc48af35a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:55.960119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 625/1260] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:45:55 2022] GET /identity/v3/role_assignments?user.id=997348872c3241f4a14c6b2fe9b66211&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:58.163458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735cddd-76a7-4dd6-b250-fe8eb8a240fb None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.164026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735cddd-76a7-4dd6-b250-fe8eb8a240fb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.164203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735cddd-76a7-4dd6-b250-fe8eb8a240fb None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.166309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 636/1261] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:45:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:58.173605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dac298-16b9-44ee-8470-876e9187f0d7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.174064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dac298-16b9-44ee-8470-876e9187f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.174174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dac298-16b9-44ee-8470-876e9187f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.192382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a7dac298-16b9-44ee-8470-876e9187f0d7 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:58.193621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7dac298-16b9-44ee-8470-876e9187f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.264068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 626/1262] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:45:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:45:58.287416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735cddd-76a7-4dd6-b250-fe8eb8a240fb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.300283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:58.393863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735cddd-76a7-4dd6-b250-fe8eb8a240fb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.436603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.437452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.437732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.438006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.448461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:45:58.456069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:58.457094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:45:58.458154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:58.462220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 637/1263] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:58.465551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dac298-16b9-44ee-8470-876e9187f0d7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.482131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:58.569822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dac298-16b9-44ee-8470-876e9187f0d7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.605031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.605733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.606016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.606294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.607063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:58.607385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.608758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.617445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 627/1264] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:58.623402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.691689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:58.728787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6832c-ab4f-4658-b209-6b63a28c64bf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.764951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.765842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.766136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.766456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.777705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.779995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:58.797788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:58.798918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:58.800243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:58.800243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:58.800243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:58.800243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:58.800243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 638/1265] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:58.804788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.889256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:58.909651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31edea-484f-4e7c-b63b-c47b33412e96 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.944739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.945470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.945886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.946158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.946998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:58.947328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.949076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.967260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 628/1266] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:58.973269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.082249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2856f50-8971-441d-b9fc-5fa192ed3e2b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.101725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:59.116461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.117310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.117598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.117875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.126097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.127545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:45:59.142234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:59.143475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:59.144477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 639/1267] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:59.149290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.250577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c98b2-5877-4e6c-9ce2-4dab0b46b29f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.283419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.284087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.284365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.284635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.285498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:59.285801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.287210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.297635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:59.306454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 629/1268] 10.222.0.91 () {56 vars in 1143 bytes} [Thu May 26 13:45:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:59.312124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.410962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9ac4-4433-4312-a981-7bcfbe677c90 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.447408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.449029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.449353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.449639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.462531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.463991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:45:59.481502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:59.483063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:59.484081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:45:59.484081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:45:59.484081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:45:59.484081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:59.484081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 640/1269] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:59.489021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.588848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c4e19-0dc8-4c24-ae57-13023d30660c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.622514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.623763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.624044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.624315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.625031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:59.625363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.626722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.638028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 630/1270] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:59.644154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.675662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:59.745734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5ae50-b03e-4375-9cd2-4be778777f52 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.780733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.781558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.781841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.782113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.789011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.790569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:45:59.807371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:45:59.808545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:45:59.809567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 641/1271] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:45:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:45:59.814638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.883261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:59.925667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f21c01-b35a-400a-9886-558e50aa3518 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.959949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cf2e5-0505-4a95-9190-58d0e179accb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.960526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf2e5-0505-4a95-9190-58d0e179accb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.960833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf2e5-0505-4a95-9190-58d0e179accb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.961096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf2e5-0505-4a95-9190-58d0e179accb None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.961889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cf2e5-0505-4a95-9190-58d0e179accb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:59.962196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cf2e5-0505-4a95-9190-58d0e179accb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.963617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cf2e5-0505-4a95-9190-58d0e179accb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.980234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 631/1272] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:45:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:59.986719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.087837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:00.097708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655700b-6f22-49b9-aa52-2dff7e084e7f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.131081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd341a6d-ae0c-4643-ba29-070fe237f9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.131969 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd341a6d-ae0c-4643-ba29-070fe237f9cc None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.132245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd341a6d-ae0c-4643-ba29-070fe237f9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.132516 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd341a6d-ae0c-4643-ba29-070fe237f9cc None admin] PATH_INFO: `/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/997348872c3241f4a14c6b2fe9b66211/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.134534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd341a6d-ae0c-4643-ba29-070fe237f9cc None admin] RBAC: Authorizing `identity:create_grant(project_id=061d1d45f0894be3b0165e8360b0035c, user_id=997348872c3241f4a14c6b2fe9b66211, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.135907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd341a6d-ae0c-4643-ba29-070fe237f9cc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.149892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 642/1273] 10.222.0.91 () {58 vars in 1439 bytes} [Thu May 26 13:45:59 2022] PUT /identity/v3/projects/061d1d45f0894be3b0165e8360b0035c/users/997348872c3241f4a14c6b2fe9b66211/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:02.404293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895881cf-468c-41ef-b85f-2681d1537ac6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.404828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895881cf-468c-41ef-b85f-2681d1537ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.404828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895881cf-468c-41ef-b85f-2681d1537ac6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.406247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 632/1274] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:46:02.411412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a26b-f59a-4094-9b64-0890b6d87cf9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.411835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a26b-f59a-4094-9b64-0890b6d87cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.411965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a26b-f59a-4094-9b64-0890b6d87cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.429208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3e50a26b-f59a-4094-9b64-0890b6d87cf9 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:02.430398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3e50a26b-f59a-4094-9b64-0890b6d87cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:02.495685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 643/1275] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:46:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:46:02.519258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895881cf-468c-41ef-b85f-2681d1537ac6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.618981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895881cf-468c-41ef-b85f-2681d1537ac6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.636205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:02.653968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.654797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.655075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.655347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.665631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:46:02.674045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:02.675343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:02.676253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 633/1276] 10.222.0.91 () {56 vars in 1096 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:02.681326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a26b-f59a-4094-9b64-0890b6d87cf9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.766606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:02.786598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a26b-f59a-4094-9b64-0890b6d87cf9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.821180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.821875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.822162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.822441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.823194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:02.823509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.824889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.833402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 644/1277] 10.222.0.91 () {56 vars in 1099 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:02.839248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.938146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b62036-31a4-4b36-bf71-d4510f9fb4a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.972949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.973969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.974292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.974594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.988734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.990481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:46:03.011087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:03.012194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:03.013143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:46:03.013143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:46:03.013143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:46:03.013143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:03.013143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 634/1278] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:03.020214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.119587 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df316d-105f-4b57-90a0-3fde33f83fee None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.160271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.160938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.161250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.161528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.162241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:03.162547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.163914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.175723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 645/1279] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:03.181428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.212566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:03.280827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412c29a-838f-4f63-8014-694c1bdcbb39 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.313496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.314293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.314578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.314857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.322248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.323685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:46:03.338762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:03.339886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:03.341079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 635/1280] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:03.346361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.350105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:03.444365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694a580-4372-4512-9cc0-2b7bba6c78a5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.477860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.478533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.478821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.479102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.479932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:03.480244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.481724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.499743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 646/1281] 10.222.0.91 () {56 vars in 1143 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:03.505523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.594928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:03.604362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6f050-7ffb-44bc-8039-d6d79467db9b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.637037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.637701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.637985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.638257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.705231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.706688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.795974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:03.806810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 636/1282] 10.222.0.91 () {58 vars in 1282 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:46:03.813383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.874278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:03.912558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a871f6-77e8-4215-9969-886aa1f05e7a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.948006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.948810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.949096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.949405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.959446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.960896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:46:03.978031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:03.979095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:03.980202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:46:03.980202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:46:03.980202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:46:03.980202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:46:03.980202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 647/1283] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:03.985514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.082741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510eae9-4fc5-40c2-aae8-8e576521d429 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.116361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.117019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.117327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.117597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.118314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:04.118615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.119972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.133042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 637/1284] 10.222.0.91 () {56 vars in 1109 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:04.138627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.240243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c2d52-1bf8-40f3-b01c-d71a104e4ad2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.254777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:04.274727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.275515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.275793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.276063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.283395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.284830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:46:04.299229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:04.300302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application May 26 13:46:04.301584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 648/1285] 10.222.0.91 () {56 vars in 1111 bytes} [Thu May 26 13:46:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:04.306341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.356057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:04.405027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96e51-28ba-4bae-9ac6-c1c2dd137e1d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.437651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc473e7f-84f2-4180-a88c-966118168462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.438307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc473e7f-84f2-4180-a88c-966118168462 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.438590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc473e7f-84f2-4180-a88c-966118168462 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.438863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc473e7f-84f2-4180-a88c-966118168462 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.439643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc473e7f-84f2-4180-a88c-966118168462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:04.439949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc473e7f-84f2-4180-a88c-966118168462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.441421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc473e7f-84f2-4180-a88c-966118168462 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.453431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 638/1286] 10.222.0.91 () {56 vars in 1148 bytes} [Thu May 26 13:46:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:04.459702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.559165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf87f8-b5e4-4432-b717-15c92511faeb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.591389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0760281-0fd9-4922-bc6e-cda292f176cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.592712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760281-0fd9-4922-bc6e-cda292f176cb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.592985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760281-0fd9-4922-bc6e-cda292f176cb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.593292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760281-0fd9-4922-bc6e-cda292f176cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.594138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760281-0fd9-4922-bc6e-cda292f176cb None admin] RBAC: Adding query filter params (role.id=2e9eb416245242c8aa082ba62b6ff1b0, scope.project.id=061d1d45f0894be3b0165e8360b0035c, user.id=997348872c3241f4a14c6b2fe9b66211) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:04.594448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760281-0fd9-4922-bc6e-cda292f176cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.595881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760281-0fd9-4922-bc6e-cda292f176cb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.601577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 649/1287] 10.222.0.91 () {56 vars in 1374 bytes} [Thu May 26 13:46:04 2022] GET /identity/v3/role_assignments?user.id=997348872c3241f4a14c6b2fe9b66211&scope.project.id=061d1d45f0894be3b0165e8360b0035c&role.id=2e9eb416245242c8aa082ba62b6ff1b0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:06.868074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6927089-736d-4faf-8d22-87c84f888fd6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.868639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6927089-736d-4faf-8d22-87c84f888fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.868639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6927089-736d-4faf-8d22-87c84f888fd6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.870042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 639/1288] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:46:06.875828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c126b5-10b1-4027-b843-e7bb045de609 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.876252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c126b5-10b1-4027-b843-e7bb045de609 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.876399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c126b5-10b1-4027-b843-e7bb045de609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.892812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-97c126b5-10b1-4027-b843-e7bb045de609 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:06.894032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-97c126b5-10b1-4027-b843-e7bb045de609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.922991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:06.964149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 650/1289] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:46:06.996852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6927089-736d-4faf-8d22-87c84f888fd6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.101809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6927089-736d-4faf-8d22-87c84f888fd6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.112594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:07.136481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.137239 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.137528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.137806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.138494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.139870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:07.152004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:07.153255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:07.154319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 640/1290] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:46:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:07.159223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c126b5-10b1-4027-b843-e7bb045de609 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.257431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c126b5-10b1-4027-b843-e7bb045de609 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.290735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acd0f1-3cd8-4096-87dc-0ccbd28bb97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.291395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd0f1-3cd8-4096-87dc-0ccbd28bb97b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.291680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd0f1-3cd8-4096-87dc-0ccbd28bb97b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.291957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd0f1-3cd8-4096-87dc-0ccbd28bb97b None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.292655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd0f1-3cd8-4096-87dc-0ccbd28bb97b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:07.293249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd0f1-3cd8-4096-87dc-0ccbd28bb97b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.295185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd0f1-3cd8-4096-87dc-0ccbd28bb97b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.300042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 651/1291] 10.222.0.91 () {56 vars in 1116 bytes} [Thu May 26 13:46:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:07.305745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.400828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999258cc-8fa1-409d-b428-65722179a39d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.433640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33174778-4a0f-4bc5-b5b9-543e469b822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.434284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174778-4a0f-4bc5-b5b9-543e469b822f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.434563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174778-4a0f-4bc5-b5b9-543e469b822f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.434830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174778-4a0f-4bc5-b5b9-543e469b822f None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.435546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33174778-4a0f-4bc5-b5b9-543e469b822f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:07.435844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33174778-4a0f-4bc5-b5b9-543e469b822f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.437210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33174778-4a0f-4bc5-b5b9-543e469b822f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.441511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 641/1292] 10.222.0.91 () {56 vars in 1116 bytes} [Thu May 26 13:46:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:09.632208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07fd1-d6ec-488f-85bb-547facf84a5c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.632761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07fd1-d6ec-488f-85bb-547facf84a5c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.632919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07fd1-d6ec-488f-85bb-547facf84a5c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.634273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 652/1293] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:46:09.639488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33773b-4510-4aba-9021-15eaf195488d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.639941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33773b-4510-4aba-9021-15eaf195488d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.640048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33773b-4510-4aba-9021-15eaf195488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.656897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1c33773b-4510-4aba-9021-15eaf195488d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:09.658113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1c33773b-4510-4aba-9021-15eaf195488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.687112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:09.724408 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 642/1294] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:46:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:46:09.747450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d07fd1-d6ec-488f-85bb-547facf84a5c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.781608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:09.851960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d07fd1-d6ec-488f-85bb-547facf84a5c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.885236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0c129-0194-4c1f-9c80-23abe6334243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.885823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0c129-0194-4c1f-9c80-23abe6334243 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.886111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0c129-0194-4c1f-9c80-23abe6334243 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.886389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0c129-0194-4c1f-9c80-23abe6334243 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.887104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0c129-0194-4c1f-9c80-23abe6334243 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.889708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0c129-0194-4c1f-9c80-23abe6334243 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.902445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 653/1295] 10.222.0.91 () {60 vars in 1141 bytes} [Thu May 26 13:46:09 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:12.136366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd434c-9d7d-416a-96cd-bea6566e294e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.136998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd434c-9d7d-416a-96cd-bea6566e294e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.136998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd434c-9d7d-416a-96cd-bea6566e294e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.138370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 643/1296] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:46:12.144105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dacf-4edb-4435-a58f-d03891c31e31 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.144807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dacf-4edb-4435-a58f-d03891c31e31 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.145032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dacf-4edb-4435-a58f-d03891c31e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.161391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ca84dacf-4edb-4435-a58f-d03891c31e31 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:12.162556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ca84dacf-4edb-4435-a58f-d03891c31e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:12.235429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 654/1297] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:46:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:46:12.258401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd434c-9d7d-416a-96cd-bea6566e294e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:12.356323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd434c-9d7d-416a-96cd-bea6566e294e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:12.389617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966629af-76e6-4599-8d43-11af327d67af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:12.390350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966629af-76e6-4599-8d43-11af327d67af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.390635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966629af-76e6-4599-8d43-11af327d67af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.390913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966629af-76e6-4599-8d43-11af327d67af None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.391565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966629af-76e6-4599-8d43-11af327d67af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:12.392935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966629af-76e6-4599-8d43-11af327d67af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-966629af-76e6-4599-8d43-11af327d67af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:12.404682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:12.406001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:12.407043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 644/1298] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:46:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:12.411944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dacf-4edb-4435-a58f-d03891c31e31 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:12.430567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:12.511320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dacf-4edb-4435-a58f-d03891c31e31 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:12.544213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:12.545500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.545775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.546047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.546750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:12.547062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:12.548427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:12.553459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 655/1299] 10.222.0.91 () {56 vars in 1116 bytes} [Thu May 26 13:46:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:12.559523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966629af-76e6-4599-8d43-11af327d67af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:12.590330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:12.658650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966629af-76e6-4599-8d43-11af327d67af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:12.691839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee81ec6-87c8-45e1-bb13-acbc38613b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:12.692486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81ec6-87c8-45e1-bb13-acbc38613b60 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.692767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81ec6-87c8-45e1-bb13-acbc38613b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.693039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81ec6-87c8-45e1-bb13-acbc38613b60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.693855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee81ec6-87c8-45e1-bb13-acbc38613b60 None admin] RBAC: Adding query filter params (interface=public, service_id=06fb4a62600f43c48849a750080b71c4, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:12.694166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee81ec6-87c8-45e1-bb13-acbc38613b60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:12.695538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee81ec6-87c8-45e1-bb13-acbc38613b60 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:12.700437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 645/1300] 10.222.0.91 () {56 vars in 1251 bytes} [Thu May 26 13:46:12 2022] GET /identity/v3/endpoints?service_id=06fb4a62600f43c48849a750080b71c4&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:12.706102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:12.785487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:12.807364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c26df-c1a5-42c3-9706-e719fdf89fd6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:12.840359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73a7e1-1e40-46f3-9fb8-68d181c33c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:12.841022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73a7e1-1e40-46f3-9fb8-68d181c33c22 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.841873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73a7e1-1e40-46f3-9fb8-68d181c33c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.842164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73a7e1-1e40-46f3-9fb8-68d181c33c22 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.842838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73a7e1-1e40-46f3-9fb8-68d181c33c22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:12.844213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73a7e1-1e40-46f3-9fb8-68d181c33c22 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:12.850856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 656/1301] 10.222.0.91 () {56 vars in 1087 bytes} [Thu May 26 13:46:12 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:15.007863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0be58-a133-499f-b770-f91755bdc286 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.008430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0be58-a133-499f-b770-f91755bdc286 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.008860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0be58-a133-499f-b770-f91755bdc286 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.011114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 646/1302] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:46:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:46:15.019507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718940-d194-4e08-9542-f9ba241b3e89 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.019962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718940-d194-4e08-9542-f9ba241b3e89 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.020401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718940-d194-4e08-9542-f9ba241b3e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.043686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1b718940-d194-4e08-9542-f9ba241b3e89 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:15.045939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1b718940-d194-4e08-9542-f9ba241b3e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.122089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 657/1303] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:46:15.144956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0be58-a133-499f-b770-f91755bdc286 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.237623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:15.244300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0be58-a133-499f-b770-f91755bdc286 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.276597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.277348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.277627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.277895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.278539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.279924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:46:15.291373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:46:15.292792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 26 13:46:15.294057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 647/1304] 10.222.0.91 () {56 vars in 1117 bytes} [Thu May 26 13:46:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:46:15.299421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b718940-d194-4e08-9542-f9ba241b3e89 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.371646 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:15.395355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b718940-d194-4e08-9542-f9ba241b3e89 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.429337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a88826-8449-4e73-a2d2-04fa5ea0ab88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.429996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a88826-8449-4e73-a2d2-04fa5ea0ab88 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.430279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a88826-8449-4e73-a2d2-04fa5ea0ab88 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.430553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a88826-8449-4e73-a2d2-04fa5ea0ab88 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.431253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a88826-8449-4e73-a2d2-04fa5ea0ab88 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:15.431560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a88826-8449-4e73-a2d2-04fa5ea0ab88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.432923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a88826-8449-4e73-a2d2-04fa5ea0ab88 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.438614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 658/1305] 10.222.0.91 () {56 vars in 1116 bytes} [Thu May 26 13:46:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:15.444483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.540361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e58086-bc37-480e-9e7c-285f41bdb431 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.572775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b922701-c49b-4bdf-8425-c422ae480e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.573445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b922701-c49b-4bdf-8425-c422ae480e94 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.573727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b922701-c49b-4bdf-8425-c422ae480e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.573998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b922701-c49b-4bdf-8425-c422ae480e94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.574729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b922701-c49b-4bdf-8425-c422ae480e94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.576107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b922701-c49b-4bdf-8425-c422ae480e94 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.594540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 648/1306] 10.222.0.91 () {60 vars in 1144 bytes} [Thu May 26 13:46:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:25.435452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=30 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:25.590324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:20.501212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b344e-bdf3-44b5-a7ef-a3bd7ea0f62c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:20.501791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b344e-bdf3-44b5-a7ef-a3bd7ea0f62c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:20.501791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b344e-bdf3-44b5-a7ef-a3bd7ea0f62c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:20.508614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead1f4-ee3b-4c9e-bcb6-e16d461d44de None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:20.509091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead1f4-ee3b-4c9e-bcb6-e16d461d44de None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:20.509215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead1f4-ee3b-4c9e-bcb6-e16d461d44de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:20.512501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 659/1307] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:20.597919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b1ead1f4-ee3b-4c9e-bcb6-e16d461d44de None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:20.599815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b1ead1f4-ee3b-4c9e-bcb6-e16d461d44de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:20.733199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 649/1308] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:47:20.772781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f31d4-f160-4412-8362-9767acdd317c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:20.773127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f31d4-f160-4412-8362-9767acdd317c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:20.773373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f31d4-f160-4412-8362-9767acdd317c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:20.775283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 660/1309] 10.222.0.91 () {54 vars in 964 bytes} [Thu May 26 13:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:20.783797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865de2-fcda-4bfd-a1f4-06e24f029b51 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:20.784663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865de2-fcda-4bfd-a1f4-06e24f029b51 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:20.785665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865de2-fcda-4bfd-a1f4-06e24f029b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:20.911663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-64865de2-fcda-4bfd-a1f4-06e24f029b51 None None] MFA Rules not processed for user `79d624b7de50480bb4002eb23ec2b360`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:20.913141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-64865de2-fcda-4bfd-a1f4-06e24f029b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:20.937837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:21.004732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 650/1310] 10.222.0.91 () {58 vars in 1061 bytes} [Thu May 26 13:47:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:21.014883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f31d4-f160-4412-8362-9767acdd317c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:21.017101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-027f31d4-f160-4412-8362-9767acdd317c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:21.227966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f31d4-f160-4412-8362-9767acdd317c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:21.333000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28b916-3681-48b6-8ad7-f88a5cfb3c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:21.334117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b916-3681-48b6-8ad7-f88a5cfb3c25 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:21.334404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b916-3681-48b6-8ad7-f88a5cfb3c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:21.334694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b916-3681-48b6-8ad7-f88a5cfb3c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:21.336370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0a28b916-3681-48b6-8ad7-f88a5cfb3c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:21.401089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:21.460582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28b916-3681-48b6-8ad7-f88a5cfb3c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:21.462061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28b916-3681-48b6-8ad7-f88a5cfb3c25 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:21.483608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 661/1311] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:24.061565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ed32-b649-4920-b664-4c3d30ccc47d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:24.061565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ed32-b649-4920-b664-4c3d30ccc47d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:24.061565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4ed32-b649-4920-b664-4c3d30ccc47d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:24.062988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 651/1312] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:24.068719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4b626-4546-432e-bb0c-5fda817287c8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:24.069409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4b626-4546-432e-bb0c-5fda817287c8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:24.069409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4b626-4546-432e-bb0c-5fda817287c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:24.152643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-30c4b626-4546-432e-bb0c-5fda817287c8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:24.153923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-30c4b626-4546-432e-bb0c-5fda817287c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:24.305995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 662/1313] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:24.344542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ed32-b649-4920-b664-4c3d30ccc47d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:24.346530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-00f4ed32-b649-4920-b664-4c3d30ccc47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:24.508862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4ed32-b649-4920-b664-4c3d30ccc47d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:24.606813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:24.609329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b88e2c-a05b-4499-af6e-4b84bf336354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:24.609929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88e2c-a05b-4499-af6e-4b84bf336354 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:24.610178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88e2c-a05b-4499-af6e-4b84bf336354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:24.610446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88e2c-a05b-4499-af6e-4b84bf336354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:24.612504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-06b88e2c-a05b-4499-af6e-4b84bf336354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:24.747578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b88e2c-a05b-4499-af6e-4b84bf336354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:24.749031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b88e2c-a05b-4499-af6e-4b84bf336354 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:24.782789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 652/1314] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:27.332710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2797369-1846-4742-8db5-f4ce670d1951 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:27.333320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2797369-1846-4742-8db5-f4ce670d1951 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:27.333320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2797369-1846-4742-8db5-f4ce670d1951 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:27.334661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 663/1315] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:27.340051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013f75d-8ffd-418d-9cab-42028da54481 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:27.340456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013f75d-8ffd-418d-9cab-42028da54481 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:27.340641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013f75d-8ffd-418d-9cab-42028da54481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:27.406066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8013f75d-8ffd-418d-9cab-42028da54481 None None] MFA Rules not processed for user `d6b97d3592ef443885d4636f8757e02c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:27.407303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8013f75d-8ffd-418d-9cab-42028da54481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:27.459675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:27.505086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 653/1316] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:27.545787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2797369-1846-4742-8db5-f4ce670d1951 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:27.547357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c2797369-1846-4742-8db5-f4ce670d1951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:27.557348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:27.655665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2797369-1846-4742-8db5-f4ce670d1951 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:27.728070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65ef0b-f87e-4058-a3dc-fb2832bc5349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:27.728629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65ef0b-f87e-4058-a3dc-fb2832bc5349 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:27.728909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65ef0b-f87e-4058-a3dc-fb2832bc5349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:27.730282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65ef0b-f87e-4058-a3dc-fb2832bc5349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:27.731686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cb65ef0b-f87e-4058-a3dc-fb2832bc5349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:27.793332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:27.840766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65ef0b-f87e-4058-a3dc-fb2832bc5349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:27.842245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65ef0b-f87e-4058-a3dc-fb2832bc5349 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:27.859837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 664/1317] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:30.639163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1318-be0a-41c9-849d-82830729329d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:30.639692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1318-be0a-41c9-849d-82830729329d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:30.639692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1318-be0a-41c9-849d-82830729329d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:30.641051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 654/1318] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:30.646811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d108bb-123b-48bf-80e2-ad461ab2f387 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:30.647265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d108bb-123b-48bf-80e2-ad461ab2f387 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:30.647414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d108bb-123b-48bf-80e2-ad461ab2f387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:30.716759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-22d108bb-123b-48bf-80e2-ad461ab2f387 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:30.718027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-22d108bb-123b-48bf-80e2-ad461ab2f387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:30.816741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 665/1319] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:30.870759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59f1aa-8b0b-46ad-b6b1-fb733ade5a35 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:30.871045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59f1aa-8b0b-46ad-b6b1-fb733ade5a35 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:30.871322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59f1aa-8b0b-46ad-b6b1-fb733ade5a35 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:30.872707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 655/1320] 10.222.0.91 () {54 vars in 964 bytes} [Thu May 26 13:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:30.878540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837022f-0abf-4930-9d04-cfc05942b441 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:30.878832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837022f-0abf-4930-9d04-cfc05942b441 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:30.879109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837022f-0abf-4930-9d04-cfc05942b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:30.900381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:30.955389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b837022f-0abf-4930-9d04-cfc05942b441 None None] MFA Rules not processed for user `79d624b7de50480bb4002eb23ec2b360`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:30.956728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b837022f-0abf-4930-9d04-cfc05942b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:31.032826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 666/1321] 10.222.0.91 () {58 vars in 1061 bytes} [Thu May 26 13:47:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:31.042031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59f1aa-8b0b-46ad-b6b1-fb733ade5a35 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:31.043554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7b59f1aa-8b0b-46ad-b6b1-fb733ade5a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:31.144600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59f1aa-8b0b-46ad-b6b1-fb733ade5a35 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:31.217609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893a701-2e32-4948-908d-7f7ddad01c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:31.218258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893a701-2e32-4948-908d-7f7ddad01c1d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:31.218541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893a701-2e32-4948-908d-7f7ddad01c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:31.218817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893a701-2e32-4948-908d-7f7ddad01c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:31.220468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9893a701-2e32-4948-908d-7f7ddad01c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:31.227802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:31.329410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893a701-2e32-4948-908d-7f7ddad01c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:31.330843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893a701-2e32-4948-908d-7f7ddad01c1d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:31.348913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 656/1322] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:34.694424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e28f-4ec4-4148-b504-7c7cdc4f9cb1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:34.694837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e28f-4ec4-4148-b504-7c7cdc4f9cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:34.695008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e28f-4ec4-4148-b504-7c7cdc4f9cb1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:34.696553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 667/1323] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:34.702201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2a79a-15d6-4fea-8702-86bbaf5327da None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:34.702627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2a79a-15d6-4fea-8702-86bbaf5327da None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:34.702767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2a79a-15d6-4fea-8702-86bbaf5327da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:34.766791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-20a2a79a-15d6-4fea-8702-86bbaf5327da None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:34.767917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-20a2a79a-15d6-4fea-8702-86bbaf5327da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:34.825579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:34.867214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 657/1324] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:34.903407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e28f-4ec4-4148-b504-7c7cdc4f9cb1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:34.905037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-24a7e28f-4ec4-4148-b504-7c7cdc4f9cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:35.016321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e28f-4ec4-4148-b504-7c7cdc4f9cb1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:35.029707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:35.094183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389d871-8c12-4052-8b60-3cb79306971e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:35.094834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d871-8c12-4052-8b60-3cb79306971e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:35.095110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d871-8c12-4052-8b60-3cb79306971e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:35.095380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d871-8c12-4052-8b60-3cb79306971e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:35.097026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0389d871-8c12-4052-8b60-3cb79306971e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:35.209052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389d871-8c12-4052-8b60-3cb79306971e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:35.210542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389d871-8c12-4052-8b60-3cb79306971e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:35.230616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 668/1325] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:37.487702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec09f4a-64a4-4981-8035-67492e30625d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:37.488233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec09f4a-64a4-4981-8035-67492e30625d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:37.488233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec09f4a-64a4-4981-8035-67492e30625d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:37.489583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 658/1326] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:37.495473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70793-d154-4410-a094-98390c35515d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:37.496015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70793-d154-4410-a094-98390c35515d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:37.496015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d70793-d154-4410-a094-98390c35515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:37.544829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:37.563437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-25d70793-d154-4410-a094-98390c35515d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:37.564791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-25d70793-d154-4410-a094-98390c35515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:37.662596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 669/1327] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:37.699492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec09f4a-64a4-4981-8035-67492e30625d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:37.701023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ec09f4a-64a4-4981-8035-67492e30625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:37.803162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec09f4a-64a4-4981-8035-67492e30625d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:37.875651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c1cca-496e-4d55-b854-9153b0e6390b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:37.876298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c1cca-496e-4d55-b854-9153b0e6390b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:37.876586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c1cca-496e-4d55-b854-9153b0e6390b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:37.876862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c1cca-496e-4d55-b854-9153b0e6390b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:37.878552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f05c1cca-496e-4d55-b854-9153b0e6390b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:37.894880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:37.988499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c1cca-496e-4d55-b854-9153b0e6390b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:37.989999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c1cca-496e-4d55-b854-9153b0e6390b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:38.007739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 659/1328] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:40.275102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03209fa3-238a-4902-bd82-5baa33a02cee None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:40.276156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03209fa3-238a-4902-bd82-5baa33a02cee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:40.276156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03209fa3-238a-4902-bd82-5baa33a02cee None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:40.277859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 670/1329] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:40.283364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d368eaa-9a50-4f03-9780-8d8b88fd983a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:40.283808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d368eaa-9a50-4f03-9780-8d8b88fd983a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:40.284058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d368eaa-9a50-4f03-9780-8d8b88fd983a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:40.349746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0d368eaa-9a50-4f03-9780-8d8b88fd983a None None] MFA Rules not processed for user `d6b97d3592ef443885d4636f8757e02c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:40.350947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d368eaa-9a50-4f03-9780-8d8b88fd983a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:40.420544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:40.454700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 660/1330] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:40.491371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03209fa3-238a-4902-bd82-5baa33a02cee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:40.492934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-03209fa3-238a-4902-bd82-5baa33a02cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:40.601632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03209fa3-238a-4902-bd82-5baa33a02cee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:40.622063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:40.677054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35d1c0-f79b-427f-83a5-bf3628506173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:40.677767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35d1c0-f79b-427f-83a5-bf3628506173 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:40.678062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35d1c0-f79b-427f-83a5-bf3628506173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:40.678337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35d1c0-f79b-427f-83a5-bf3628506173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:40.679986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5f35d1c0-f79b-427f-83a5-bf3628506173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:40.793114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35d1c0-f79b-427f-83a5-bf3628506173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:40.794522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35d1c0-f79b-427f-83a5-bf3628506173 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:40.813243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 671/1331] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:43.276325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12875-c46a-4e5f-a8b0-5248c632194a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:43.276916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12875-c46a-4e5f-a8b0-5248c632194a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:43.276916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12875-c46a-4e5f-a8b0-5248c632194a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:43.278310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 661/1332] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:43.283896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98183b8c-8a1f-4326-983e-2ad06af753e0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:43.284407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98183b8c-8a1f-4326-983e-2ad06af753e0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:43.284554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98183b8c-8a1f-4326-983e-2ad06af753e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:43.352829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:43.355608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-98183b8c-8a1f-4326-983e-2ad06af753e0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:43.357432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-98183b8c-8a1f-4326-983e-2ad06af753e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:43.453765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 672/1333] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:47:43.488735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d12875-c46a-4e5f-a8b0-5248c632194a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:43.490404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f3d12875-c46a-4e5f-a8b0-5248c632194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:43.597627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d12875-c46a-4e5f-a8b0-5248c632194a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:43.672470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0624c06-e881-42b2-bd0d-bcec2cd6b179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:43.673124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624c06-e881-42b2-bd0d-bcec2cd6b179 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:43.673444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624c06-e881-42b2-bd0d-bcec2cd6b179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:43.673724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624c06-e881-42b2-bd0d-bcec2cd6b179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:43.675381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b0624c06-e881-42b2-bd0d-bcec2cd6b179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:43.702753 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:43.786260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0624c06-e881-42b2-bd0d-bcec2cd6b179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:43.787690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0624c06-e881-42b2-bd0d-bcec2cd6b179 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:43.805949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 662/1334] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:46.091900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f0a1-1e1b-4119-b340-41d4ba237e55 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:46.092474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f0a1-1e1b-4119-b340-41d4ba237e55 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:46.092474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f0a1-1e1b-4119-b340-41d4ba237e55 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:46.093909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 673/1335] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:46.099051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3164818-f185-4c38-96a1-00614e5fa9cb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:46.099494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3164818-f185-4c38-96a1-00614e5fa9cb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:46.099611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3164818-f185-4c38-96a1-00614e5fa9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:46.163684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d3164818-f185-4c38-96a1-00614e5fa9cb None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:46.164901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d3164818-f185-4c38-96a1-00614e5fa9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:46.238862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:46.262234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 663/1336] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:46.301087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f0a1-1e1b-4119-b340-41d4ba237e55 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:46.302679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-32b9f0a1-1e1b-4119-b340-41d4ba237e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:46.406261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f0a1-1e1b-4119-b340-41d4ba237e55 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:46.437917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:46.483198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e5263-2e71-43aa-8e8c-072072eb9844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:46.483848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e5263-2e71-43aa-8e8c-072072eb9844 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:46.484194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e5263-2e71-43aa-8e8c-072072eb9844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:46.484508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e5263-2e71-43aa-8e8c-072072eb9844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:46.486187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-438e5263-2e71-43aa-8e8c-072072eb9844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:46.594068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e5263-2e71-43aa-8e8c-072072eb9844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:46.595522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e5263-2e71-43aa-8e8c-072072eb9844 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:46.613629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 674/1337] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:49.215294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286944-9389-4d1d-8cfc-5bbf1188cc42 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:49.215746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286944-9389-4d1d-8cfc-5bbf1188cc42 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:49.215840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286944-9389-4d1d-8cfc-5bbf1188cc42 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:49.217253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 664/1338] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:49.222563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354188d9-5fd3-4846-92dd-9fe723672ecd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:49.223005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354188d9-5fd3-4846-92dd-9fe723672ecd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:49.223109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354188d9-5fd3-4846-92dd-9fe723672ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:49.287352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-354188d9-5fd3-4846-92dd-9fe723672ecd None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:49.288565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-354188d9-5fd3-4846-92dd-9fe723672ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:49.297725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:49.384863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 675/1339] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:49.421647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78286944-9389-4d1d-8cfc-5bbf1188cc42 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:49.423198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78286944-9389-4d1d-8cfc-5bbf1188cc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:49.527534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78286944-9389-4d1d-8cfc-5bbf1188cc42 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:49.604935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31037a0-e346-4138-8f9a-32158ec45b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:49.605619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31037a0-e346-4138-8f9a-32158ec45b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:49.605900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31037a0-e346-4138-8f9a-32158ec45b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:49.606170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31037a0-e346-4138-8f9a-32158ec45b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:49.607872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d31037a0-e346-4138-8f9a-32158ec45b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:49.644272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:49.721980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31037a0-e346-4138-8f9a-32158ec45b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:49.723420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31037a0-e346-4138-8f9a-32158ec45b2a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:49.745905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 665/1340] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:54.061724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec80c3-4fac-4a17-9682-6d6737458b0f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.062304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec80c3-4fac-4a17-9682-6d6737458b0f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.062304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec80c3-4fac-4a17-9682-6d6737458b0f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.063655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 676/1341] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:54.068871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee59a86-2ac8-4c7d-baba-d207b9799402 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.069336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee59a86-2ac8-4c7d-baba-d207b9799402 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.069525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee59a86-2ac8-4c7d-baba-d207b9799402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.139399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ee59a86-2ac8-4c7d-baba-d207b9799402 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:54.140603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ee59a86-2ac8-4c7d-baba-d207b9799402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:54.242798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:54.256336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 666/1342] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:54.295313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec80c3-4fac-4a17-9682-6d6737458b0f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:54.297458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-84ec80c3-4fac-4a17-9682-6d6737458b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:54.393038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec80c3-4fac-4a17-9682-6d6737458b0f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:54.396570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73f5e0-c011-489a-9885-2f52db452b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:54.397248 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73f5e0-c011-489a-9885-2f52db452b00 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.397531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73f5e0-c011-489a-9885-2f52db452b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.397804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73f5e0-c011-489a-9885-2f52db452b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.407790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73f5e0-c011-489a-9885-2f52db452b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:54.409258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73f5e0-c011-489a-9885-2f52db452b00 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:54.420870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 677/1343] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:57.308155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95e570-5238-4b39-a2fc-db2640107cd7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:57.308662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95e570-5238-4b39-a2fc-db2640107cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:57.308662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95e570-5238-4b39-a2fc-db2640107cd7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:57.310231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 667/1344] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:47:57.315401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a984989-d1e2-4377-b7dd-fd8944ca76b8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:57.315849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a984989-d1e2-4377-b7dd-fd8944ca76b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:57.316051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a984989-d1e2-4377-b7dd-fd8944ca76b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:57.345634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4a984989-d1e2-4377-b7dd-fd8944ca76b8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:57.346853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4a984989-d1e2-4377-b7dd-fd8944ca76b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:57.374961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:57.425578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 678/1345] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:47:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:57.461827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95e570-5238-4b39-a2fc-db2640107cd7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:57.472961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95e570-5238-4b39-a2fc-db2640107cd7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:57.476506 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5ab41-b432-45b7-afe3-e47bf0f71f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:57.477255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5ab41-b432-45b7-afe3-e47bf0f71f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:57.477607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5ab41-b432-45b7-afe3-e47bf0f71f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:57.477979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5ab41-b432-45b7-afe3-e47bf0f71f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:57.486962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5ab41-b432-45b7-afe3-e47bf0f71f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:57.488400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5ab41-b432-45b7-afe3-e47bf0f71f1c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:57.498504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 668/1346] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:47:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:01.850202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fe474-944f-4653-962f-1a1f79133a05 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:01.850825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fe474-944f-4653-962f-1a1f79133a05 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:01.850825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fe474-944f-4653-962f-1a1f79133a05 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:01.852116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 679/1347] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:01.858042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c32570-312f-4801-a79a-481b53749a7b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:01.858489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c32570-312f-4801-a79a-481b53749a7b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:01.858815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c32570-312f-4801-a79a-481b53749a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:01.882339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-88c32570-312f-4801-a79a-481b53749a7b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:01.883442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-88c32570-312f-4801-a79a-481b53749a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:01.891340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 669/1348] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:01.930973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fe474-944f-4653-962f-1a1f79133a05 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:01.942035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fe474-944f-4653-962f-1a1f79133a05 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:01.945989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc2f07-0404-4852-8267-1375fb4f7940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:01.947308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc2f07-0404-4852-8267-1375fb4f7940 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:01.947683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc2f07-0404-4852-8267-1375fb4f7940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:01.948060 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc2f07-0404-4852-8267-1375fb4f7940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:01.957955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc2f07-0404-4852-8267-1375fb4f7940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:01.959395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc2f07-0404-4852-8267-1375fb4f7940 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:01.969636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 680/1349] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:07.898753 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e7501-6e1c-4b3a-b5cb-af0d6335f593 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:07.900552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e7501-6e1c-4b3a-b5cb-af0d6335f593 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:07.901280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e7501-6e1c-4b3a-b5cb-af0d6335f593 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:07.903747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 670/1350] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:48:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:07.908595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2d8df-bb60-4a3d-a18c-febf50253750 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:07.909217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2d8df-bb60-4a3d-a18c-febf50253750 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:07.909217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2d8df-bb60-4a3d-a18c-febf50253750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:08.021047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-27c2d8df-bb60-4a3d-a18c-febf50253750 None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:08.022626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-27c2d8df-bb60-4a3d-a18c-febf50253750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:08.128618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 681/1351] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:48:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:11.876971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:13.304737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b81619-ef55-414b-b34a-7607c6f85ba6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:13.306830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b81619-ef55-414b-b34a-7607c6f85ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:13.307612 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b81619-ef55-414b-b34a-7607c6f85ba6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:13.309796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 671/1352] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:48:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:13.315593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb2ff2-d33f-403d-bfc8-4e0313dff596 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:13.316081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb2ff2-d33f-403d-bfc8-4e0313dff596 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:13.316161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb2ff2-d33f-403d-bfc8-4e0313dff596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:13.336002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6fcb2ff2-d33f-403d-bfc8-4e0313dff596 None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:13.337851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6fcb2ff2-d33f-403d-bfc8-4e0313dff596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:13.346764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 682/1353] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:48:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:14.752004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba816-be3c-492c-b175-567278e3d049 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:14.752627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba816-be3c-492c-b175-567278e3d049 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:14.752627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba816-be3c-492c-b175-567278e3d049 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:14.754162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 672/1354] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:14.759777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19429b91-6787-4d5f-978a-3edcab93a473 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:14.760377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19429b91-6787-4d5f-978a-3edcab93a473 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:14.760377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19429b91-6787-4d5f-978a-3edcab93a473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:14.779848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-19429b91-6787-4d5f-978a-3edcab93a473 None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:14.781208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-19429b91-6787-4d5f-978a-3edcab93a473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:14.790648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 683/1355] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:48:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:18.763990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f59443-5abd-457f-9ce2-3c2d78cb6955 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:18.764594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f59443-5abd-457f-9ce2-3c2d78cb6955 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:18.764594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f59443-5abd-457f-9ce2-3c2d78cb6955 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:18.766166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 673/1356] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:18.774648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851246c9-9dce-4fd5-8b4f-f7e934b6fb42 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:18.775911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851246c9-9dce-4fd5-8b4f-f7e934b6fb42 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:18.776669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851246c9-9dce-4fd5-8b4f-f7e934b6fb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:18.800707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-851246c9-9dce-4fd5-8b4f-f7e934b6fb42 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:18.802118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-851246c9-9dce-4fd5-8b4f-f7e934b6fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:18.812598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 684/1357] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:18.870122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458653-de43-4963-9596-b2cfbb80df25 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:18.870768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458653-de43-4963-9596-b2cfbb80df25 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:18.871148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458653-de43-4963-9596-b2cfbb80df25 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:18.873057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 674/1358] 10.222.0.91 () {54 vars in 961 bytes} [Thu May 26 13:48:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:18.889628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97fafa-3447-4e08-ba3b-74b0bc0a37ff None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:18.890174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97fafa-3447-4e08-ba3b-74b0bc0a37ff None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:18.890592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97fafa-3447-4e08-ba3b-74b0bc0a37ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:18.948843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:18.962150 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8e97fafa-3447-4e08-ba3b-74b0bc0a37ff None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:18.963263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8e97fafa-3447-4e08-ba3b-74b0bc0a37ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:19.093776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 685/1359] 10.222.0.91 () {58 vars in 1058 bytes} [Thu May 26 13:48:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:19.103446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458653-de43-4963-9596-b2cfbb80df25 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:19.126061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458653-de43-4963-9596-b2cfbb80df25 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:19.132000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c995e-98c9-4623-a18e-cae00c58b455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:19.132656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c995e-98c9-4623-a18e-cae00c58b455 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:19.132927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c995e-98c9-4623-a18e-cae00c58b455 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:19.134080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c995e-98c9-4623-a18e-cae00c58b455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:19.144967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c995e-98c9-4623-a18e-cae00c58b455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:19.147196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c995e-98c9-4623-a18e-cae00c58b455 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:19.160071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 675/1360] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:21.773476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9e57e-0caf-41c7-8195-2c953bfcff66 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:21.774282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9e57e-0caf-41c7-8195-2c953bfcff66 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:21.774282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9e57e-0caf-41c7-8195-2c953bfcff66 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:21.775660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 686/1361] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:48:21.780862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b3e54-229d-4783-8c04-67ccec27e567 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:21.781619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b3e54-229d-4783-8c04-67ccec27e567 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:21.781619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b3e54-229d-4783-8c04-67ccec27e567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:21.801788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-534b3e54-229d-4783-8c04-67ccec27e567 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:21.803011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-534b3e54-229d-4783-8c04-67ccec27e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:21.813667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 676/1362] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:21.858811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9e57e-0caf-41c7-8195-2c953bfcff66 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:21.876374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9e57e-0caf-41c7-8195-2c953bfcff66 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:21.884504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcf5d5-9f8a-4d87-b6d9-a4e21677af7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:21.885123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf5d5-9f8a-4d87-b6d9-a4e21677af7c service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:21.885440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf5d5-9f8a-4d87-b6d9-a4e21677af7c service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:21.885836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf5d5-9f8a-4d87-b6d9-a4e21677af7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:21.896253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcf5d5-9f8a-4d87-b6d9-a4e21677af7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:21.897942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcf5d5-9f8a-4d87-b6d9-a4e21677af7c service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:21.922560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 687/1363] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:24.549901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ece5-c04f-405a-a341-3fd69aebd025 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:24.550501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ece5-c04f-405a-a341-3fd69aebd025 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:24.550501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ece5-c04f-405a-a341-3fd69aebd025 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:24.551782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 677/1364] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:24.557748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f4b0-10a7-42f3-84c0-337615eaf1ee None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:24.558582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f4b0-10a7-42f3-84c0-337615eaf1ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:24.558776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f4b0-10a7-42f3-84c0-337615eaf1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:24.578511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c599f4b0-10a7-42f3-84c0-337615eaf1ee None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:24.579738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c599f4b0-10a7-42f3-84c0-337615eaf1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:24.588758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 688/1365] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:24.636431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a27c0-c803-4645-9ff7-39b395562a2a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:24.636763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a27c0-c803-4645-9ff7-39b395562a2a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:24.636987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a27c0-c803-4645-9ff7-39b395562a2a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:24.638726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 678/1366] 10.222.0.91 () {54 vars in 961 bytes} [Thu May 26 13:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:24.644506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86791-e846-4c0a-973d-942b62b5380b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:24.644790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86791-e846-4c0a-973d-942b62b5380b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:24.645063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86791-e846-4c0a-973d-942b62b5380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:24.666710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-16a86791-e846-4c0a-973d-942b62b5380b None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:24.667929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-16a86791-e846-4c0a-973d-942b62b5380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:24.677566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 689/1367] 10.222.0.91 () {58 vars in 1058 bytes} [Thu May 26 13:48:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:24.686012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a27c0-c803-4645-9ff7-39b395562a2a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:24.702537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a27c0-c803-4645-9ff7-39b395562a2a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:24.707862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b3755-60ca-4b94-ad22-9f702bfe1499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:24.708674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b3755-60ca-4b94-ad22-9f702bfe1499 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:24.708970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b3755-60ca-4b94-ad22-9f702bfe1499 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:24.709276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b3755-60ca-4b94-ad22-9f702bfe1499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:24.719336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b3755-60ca-4b94-ad22-9f702bfe1499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:24.721770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b3755-60ca-4b94-ad22-9f702bfe1499 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:24.731607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 679/1368] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:27.097468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51708ca-8d5a-4707-9e96-70ffbbc71b0e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:27.098135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51708ca-8d5a-4707-9e96-70ffbbc71b0e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:27.098135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51708ca-8d5a-4707-9e96-70ffbbc71b0e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:27.099317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 690/1369] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:27.106067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477da95-8cbd-428b-88c6-cba9d8a7d74a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:27.106637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477da95-8cbd-428b-88c6-cba9d8a7d74a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:27.106637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477da95-8cbd-428b-88c6-cba9d8a7d74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:27.127759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8477da95-8cbd-428b-88c6-cba9d8a7d74a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:27.128973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8477da95-8cbd-428b-88c6-cba9d8a7d74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:27.138563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 680/1370] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:27.185696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51708ca-8d5a-4707-9e96-70ffbbc71b0e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:27.217895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51708ca-8d5a-4707-9e96-70ffbbc71b0e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:27.229680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee96d6-57eb-43e2-beae-5fae7a38bb4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:27.230991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee96d6-57eb-43e2-beae-5fae7a38bb4f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:27.231798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee96d6-57eb-43e2-beae-5fae7a38bb4f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:27.232323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee96d6-57eb-43e2-beae-5fae7a38bb4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:27.251724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee96d6-57eb-43e2-beae-5fae7a38bb4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:27.255076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee96d6-57eb-43e2-beae-5fae7a38bb4f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:27.275532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 691/1371] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:48:27.900347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc2341-ccd1-44e4-88f4-24546271f0f4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:27.900772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc2341-ccd1-44e4-88f4-24546271f0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:27.901372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc2341-ccd1-44e4-88f4-24546271f0f4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:27.904048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 681/1372] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:27.911286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91246-3c28-4c33-b22a-fd671a8fac4d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:27.911960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91246-3c28-4c33-b22a-fd671a8fac4d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:27.912432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91246-3c28-4c33-b22a-fd671a8fac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:27.938334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-07c91246-3c28-4c33-b22a-fd671a8fac4d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:27.939651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07c91246-3c28-4c33-b22a-fd671a8fac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:27.948936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 692/1373] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:28.331986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246416a1-03d1-4fdf-ad1f-45b60b5c8c83 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:28.331986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246416a1-03d1-4fdf-ad1f-45b60b5c8c83 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:28.331986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246416a1-03d1-4fdf-ad1f-45b60b5c8c83 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:28.334489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 682/1374] 10.222.0.91 () {54 vars in 963 bytes} [Thu May 26 13:48:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:28.342478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9477ac-4902-455b-953d-4f752deddbc7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:28.343064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9477ac-4902-455b-953d-4f752deddbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:28.343064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9477ac-4902-455b-953d-4f752deddbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:28.399583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2f9477ac-4902-455b-953d-4f752deddbc7 None None] MFA Rules not processed for user `8a381dccf1ca4dacaab4e0ae0d473525`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:28.400914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2f9477ac-4902-455b-953d-4f752deddbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:28.481233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 693/1375] 10.222.0.91 () {58 vars in 1060 bytes} [Thu May 26 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:28.488814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246416a1-03d1-4fdf-ad1f-45b60b5c8c83 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:28.503450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246416a1-03d1-4fdf-ad1f-45b60b5c8c83 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:28.508145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d15c1-18ca-4f1c-b38b-4162a728e717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:28.508804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d15c1-18ca-4f1c-b38b-4162a728e717 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:28.509082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d15c1-18ca-4f1c-b38b-4162a728e717 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:28.509622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d15c1-18ca-4f1c-b38b-4162a728e717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:28.520770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d15c1-18ca-4f1c-b38b-4162a728e717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:28.522495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d15c1-18ca-4f1c-b38b-4162a728e717 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:28.534926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 683/1376] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:29.666345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be5e8-67c4-4a96-9979-53b508698c97 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:29.666886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be5e8-67c4-4a96-9979-53b508698c97 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:29.666886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be5e8-67c4-4a96-9979-53b508698c97 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:29.668253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 694/1377] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:29.673747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d174b-827a-452a-a5d3-8255613ea48b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:29.674229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d174b-827a-452a-a5d3-8255613ea48b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:29.674341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d174b-827a-452a-a5d3-8255613ea48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:29.693861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-830d174b-827a-452a-a5d3-8255613ea48b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:29.695063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-830d174b-827a-452a-a5d3-8255613ea48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:29.703917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 684/1378] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:29.749788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6be5e8-67c4-4a96-9979-53b508698c97 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:29.765490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6be5e8-67c4-4a96-9979-53b508698c97 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:29.772718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a7e36-9839-4a9b-9b2d-3d17099dea9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:29.773447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7e36-9839-4a9b-9b2d-3d17099dea9c service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:29.774319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7e36-9839-4a9b-9b2d-3d17099dea9c service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:29.774533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a7e36-9839-4a9b-9b2d-3d17099dea9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:29.783523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a7e36-9839-4a9b-9b2d-3d17099dea9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:29.785219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a7e36-9839-4a9b-9b2d-3d17099dea9c service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:29.789781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:29.797033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 695/1379] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:32.086375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a9a96-580a-4f34-944c-1d34435e8e4e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:32.086905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a9a96-580a-4f34-944c-1d34435e8e4e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:32.086999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a9a96-580a-4f34-944c-1d34435e8e4e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:32.088324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 685/1380] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:32.093594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d370f-a827-470f-8b9e-501228f5c915 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:32.094182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d370f-a827-470f-8b9e-501228f5c915 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:32.094182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d370f-a827-470f-8b9e-501228f5c915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:32.112006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1f3d370f-a827-470f-8b9e-501228f5c915 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:32.113236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1f3d370f-a827-470f-8b9e-501228f5c915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:32.121045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 696/1381] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:32.166279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a9a96-580a-4f34-944c-1d34435e8e4e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:32.185401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a9a96-580a-4f34-944c-1d34435e8e4e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:32.191623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ada61-7f28-4eb8-bf35-0333be33a1cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:32.192289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ada61-7f28-4eb8-bf35-0333be33a1cd service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:32.192574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ada61-7f28-4eb8-bf35-0333be33a1cd service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:32.192884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ada61-7f28-4eb8-bf35-0333be33a1cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:32.202525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ada61-7f28-4eb8-bf35-0333be33a1cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:32.204115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ada61-7f28-4eb8-bf35-0333be33a1cd service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:32.214462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 686/1382] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:34.558919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93318a-9811-4236-980d-c480d0a33b86 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:34.560014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93318a-9811-4236-980d-c480d0a33b86 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:34.560780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93318a-9811-4236-980d-c480d0a33b86 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:34.563214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 697/1383] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:34.576109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943791f-9d8d-47a7-a7d9-fb657785a8b5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:34.577382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943791f-9d8d-47a7-a7d9-fb657785a8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:34.577382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943791f-9d8d-47a7-a7d9-fb657785a8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:34.603251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6943791f-9d8d-47a7-a7d9-fb657785a8b5 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:34.604852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6943791f-9d8d-47a7-a7d9-fb657785a8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:34.616116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 687/1384] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:34.665312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93318a-9811-4236-980d-c480d0a33b86 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:34.680167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93318a-9811-4236-980d-c480d0a33b86 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:34.685741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6d260-ceca-4737-abd0-fdee93b3ba51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:34.686402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6d260-ceca-4737-abd0-fdee93b3ba51 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:34.686715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6d260-ceca-4737-abd0-fdee93b3ba51 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:34.687017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6d260-ceca-4737-abd0-fdee93b3ba51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:34.696210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6d260-ceca-4737-abd0-fdee93b3ba51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:34.697902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6d260-ceca-4737-abd0-fdee93b3ba51 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:34.708732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 698/1385] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:36.967499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774bb95-fa07-479e-80db-6b18ae3b799a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:36.968036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774bb95-fa07-479e-80db-6b18ae3b799a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:36.968276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774bb95-fa07-479e-80db-6b18ae3b799a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:36.970460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 688/1386] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:36.976032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546ce80-c2b1-4362-ba5d-d2cf38e386c6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:36.976640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546ce80-c2b1-4362-ba5d-d2cf38e386c6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:36.976640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546ce80-c2b1-4362-ba5d-d2cf38e386c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:36.995099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d546ce80-c2b1-4362-ba5d-d2cf38e386c6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:36.996205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d546ce80-c2b1-4362-ba5d-d2cf38e386c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:37.004278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 699/1387] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:37.049107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774bb95-fa07-479e-80db-6b18ae3b799a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:37.068719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774bb95-fa07-479e-80db-6b18ae3b799a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:37.076673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71f33b-81d0-42d7-a6d8-e31bf0395c2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:37.077600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f33b-81d0-42d7-a6d8-e31bf0395c2d service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:37.077978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f33b-81d0-42d7-a6d8-e31bf0395c2d service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:37.078342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f33b-81d0-42d7-a6d8-e31bf0395c2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:37.090051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71f33b-81d0-42d7-a6d8-e31bf0395c2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:37.092269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71f33b-81d0-42d7-a6d8-e31bf0395c2d service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:37.104884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 689/1388] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:39.361372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674aa26-ce5c-433a-86fa-c73f66158b45 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:39.361999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674aa26-ce5c-433a-86fa-c73f66158b45 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:39.361999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674aa26-ce5c-433a-86fa-c73f66158b45 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:39.363410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 700/1389] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:39.369080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede899b-7db1-4245-84bd-0ba80efdb013 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:39.369695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede899b-7db1-4245-84bd-0ba80efdb013 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:39.369695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede899b-7db1-4245-84bd-0ba80efdb013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:39.392395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dede899b-7db1-4245-84bd-0ba80efdb013 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:39.394058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dede899b-7db1-4245-84bd-0ba80efdb013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:39.409220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 690/1390] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:39.455383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674aa26-ce5c-433a-86fa-c73f66158b45 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:39.470818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674aa26-ce5c-433a-86fa-c73f66158b45 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:39.476683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b859495-8869-4e2f-980c-4562f16f841e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:39.477370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859495-8869-4e2f-980c-4562f16f841e service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:39.477710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859495-8869-4e2f-980c-4562f16f841e service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:39.477999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859495-8869-4e2f-980c-4562f16f841e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:39.487258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b859495-8869-4e2f-980c-4562f16f841e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:39.488871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b859495-8869-4e2f-980c-4562f16f841e service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:39.499389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 701/1391] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:41.748572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4b85-51bd-4ab4-8bae-bad241711bcc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:41.749110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4b85-51bd-4ab4-8bae-bad241711bcc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:41.749483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d4b85-51bd-4ab4-8bae-bad241711bcc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:41.751292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 691/1392] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:41.756419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549c5de-0677-4e0e-80f6-6c647d17b058 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:41.757103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549c5de-0677-4e0e-80f6-6c647d17b058 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:41.757103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549c5de-0677-4e0e-80f6-6c647d17b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:41.776135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f549c5de-0677-4e0e-80f6-6c647d17b058 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:41.777310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f549c5de-0677-4e0e-80f6-6c647d17b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:41.785107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 702/1393] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:41.828707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4b85-51bd-4ab4-8bae-bad241711bcc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:41.847669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d4b85-51bd-4ab4-8bae-bad241711bcc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:41.854736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7949c19-dfa8-4621-ac43-84dbeda3ec25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:41.855623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7949c19-dfa8-4621-ac43-84dbeda3ec25 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:41.855997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7949c19-dfa8-4621-ac43-84dbeda3ec25 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:41.856363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7949c19-dfa8-4621-ac43-84dbeda3ec25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:41.867268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7949c19-dfa8-4621-ac43-84dbeda3ec25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:41.869345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7949c19-dfa8-4621-ac43-84dbeda3ec25 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:41.881486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 692/1394] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:44.119710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe37846-f0c4-4e4e-8d3e-f0c1c2ef05b7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.120288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe37846-f0c4-4e4e-8d3e-f0c1c2ef05b7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.120288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe37846-f0c4-4e4e-8d3e-f0c1c2ef05b7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.121576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 703/1395] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:44.126968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d23917-cb89-4d1b-9da4-d1af4d291d4d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.127405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d23917-cb89-4d1b-9da4-d1af4d291d4d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.127508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d23917-cb89-4d1b-9da4-d1af4d291d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.133630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=52 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:44.146188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-91d23917-cb89-4d1b-9da4-d1af4d291d4d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:44.147394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-91d23917-cb89-4d1b-9da4-d1af4d291d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:44.155274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 693/1396] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:44.199867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe37846-f0c4-4e4e-8d3e-f0c1c2ef05b7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:44.214414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe37846-f0c4-4e4e-8d3e-f0c1c2ef05b7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:44.219775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e63405-5eb9-40cb-b2af-280009859baf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:44.220447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e63405-5eb9-40cb-b2af-280009859baf service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.220734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e63405-5eb9-40cb-b2af-280009859baf service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.221019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e63405-5eb9-40cb-b2af-280009859baf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.230567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e63405-5eb9-40cb-b2af-280009859baf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:44.232167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e63405-5eb9-40cb-b2af-280009859baf service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:44.242322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 704/1397] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:44.560643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea235-a34a-409c-b63f-5c7555f00263 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.561005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea235-a34a-409c-b63f-5c7555f00263 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.561289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea235-a34a-409c-b63f-5c7555f00263 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.562780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 694/1398] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:44.567872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2ff1-2c6a-4d34-a612-00c1cb069ada None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.568177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2ff1-2c6a-4d34-a612-00c1cb069ada None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.568448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2ff1-2c6a-4d34-a612-00c1cb069ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.593105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5efb2ff1-2c6a-4d34-a612-00c1cb069ada None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:44.594360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5efb2ff1-2c6a-4d34-a612-00c1cb069ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:44.603705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 705/1399] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:44.665402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761366-137d-4d1c-804e-f3cea03d5211 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.665834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761366-137d-4d1c-804e-f3cea03d5211 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.666246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761366-137d-4d1c-804e-f3cea03d5211 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.668372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 695/1400] 10.222.0.91 () {54 vars in 962 bytes} [Thu May 26 13:48:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:44.674715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d4586-7880-49c3-a191-a2681b3b4ff9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.675092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d4586-7880-49c3-a191-a2681b3b4ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.675305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d4586-7880-49c3-a191-a2681b3b4ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.727676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7a1d4586-7880-49c3-a191-a2681b3b4ff9 None None] MFA Rules not processed for user `a0c209755dfc4b4089176a056cf2f683`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:44.729358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7a1d4586-7880-49c3-a191-a2681b3b4ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:44.789988 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 706/1401] 10.222.0.91 () {58 vars in 1059 bytes} [Thu May 26 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:44.797602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e761366-137d-4d1c-804e-f3cea03d5211 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:44.808364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e761366-137d-4d1c-804e-f3cea03d5211 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:44.812854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ca014-d301-4231-8fb1-427b95cd2164 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:44.813523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ca014-d301-4231-8fb1-427b95cd2164 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:44.813807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ca014-d301-4231-8fb1-427b95cd2164 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:44.814087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ca014-d301-4231-8fb1-427b95cd2164 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:44.823711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ca014-d301-4231-8fb1-427b95cd2164 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:44.825164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ca014-d301-4231-8fb1-427b95cd2164 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:44.834767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 696/1402] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:48:45.404003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aac94-b648-4c1b-a47c-f090e159d2ef None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:45.404515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aac94-b648-4c1b-a47c-f090e159d2ef None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:45.404629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aac94-b648-4c1b-a47c-f090e159d2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:45.458270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-685aac94-b648-4c1b-a47c-f090e159d2ef None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:45.459485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-685aac94-b648-4c1b-a47c-f090e159d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:45.501485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:45.526572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 707/1403] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:45.573304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc588f-c986-4f7e-a9d1-618a3cc3b68d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:45.573945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc588f-c986-4f7e-a9d1-618a3cc3b68d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:45.574048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc588f-c986-4f7e-a9d1-618a3cc3b68d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:45.575486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 697/1404] 10.222.0.91 () {54 vars in 942 bytes} [Thu May 26 13:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:45.582209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3c54d-9417-44bf-9739-8a8c0c59a08e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:45.582720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3c54d-9417-44bf-9739-8a8c0c59a08e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:45.582853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3c54d-9417-44bf-9739-8a8c0c59a08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:45.634804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5cb3c54d-9417-44bf-9739-8a8c0c59a08e None None] MFA Rules not processed for user `0cbe226d7639456ebb94de533de30d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:45.636241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5cb3c54d-9417-44bf-9739-8a8c0c59a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:45.717583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 708/1405] 10.222.0.91 () {58 vars in 1039 bytes} [Thu May 26 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:45.727160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc588f-c986-4f7e-a9d1-618a3cc3b68d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:45.742163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc588f-c986-4f7e-a9d1-618a3cc3b68d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:45.747267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98dd31-fa59-4873-b21c-8ab5ab6824ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:45.747906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98dd31-fa59-4873-b21c-8ab5ab6824ee service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:45.748183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98dd31-fa59-4873-b21c-8ab5ab6824ee service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:45.748652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98dd31-fa59-4873-b21c-8ab5ab6824ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:45.759994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98dd31-fa59-4873-b21c-8ab5ab6824ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:45.762776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98dd31-fa59-4873-b21c-8ab5ab6824ee service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:45.771193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 698/1406] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 13:48:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:48:46.574279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b999c5-2f93-4662-a452-0cd829f6d079 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:46.574809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b999c5-2f93-4662-a452-0cd829f6d079 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:46.574809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b999c5-2f93-4662-a452-0cd829f6d079 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:46.576140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 709/1407] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:48:46.581438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae7b08-02ea-4444-8e16-12c02b728cac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:46.581857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae7b08-02ea-4444-8e16-12c02b728cac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:46.582000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae7b08-02ea-4444-8e16-12c02b728cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:46.601760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-94ae7b08-02ea-4444-8e16-12c02b728cac None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:46.602959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-94ae7b08-02ea-4444-8e16-12c02b728cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:46.611332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 699/1408] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:46.656787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b999c5-2f93-4662-a452-0cd829f6d079 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:46.675564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b999c5-2f93-4662-a452-0cd829f6d079 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:46.681281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a865d45-f232-4ce9-b759-d88f533be865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:46.682159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865d45-f232-4ce9-b759-d88f533be865 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:46.682514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865d45-f232-4ce9-b759-d88f533be865 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:46.682865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865d45-f232-4ce9-b759-d88f533be865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:46.691703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a865d45-f232-4ce9-b759-d88f533be865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:46.693475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a865d45-f232-4ce9-b759-d88f533be865 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:46.703119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 710/1409] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:56.595012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=17 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:56.698809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=59 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:09.781652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b088d0-7fcf-4687-bd5b-ae6b53cbc05a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:09.781652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b088d0-7fcf-4687-bd5b-ae6b53cbc05a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:09.783044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b088d0-7fcf-4687-bd5b-ae6b53cbc05a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:09.783765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 700/1410] 10.222.0.91 () {54 vars in 918 bytes} [Thu May 26 13:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:09.788788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bbe08-0e39-4e3e-be20-fd39db73fd6a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:09.789346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bbe08-0e39-4e3e-be20-fd39db73fd6a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:09.789346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bbe08-0e39-4e3e-be20-fd39db73fd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:09.808080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c47bbe08-0e39-4e3e-be20-fd39db73fd6a None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:09.809334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c47bbe08-0e39-4e3e-be20-fd39db73fd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:09.818096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 711/1411] 10.222.0.91 () {58 vars in 1015 bytes} [Thu May 26 13:49:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:09.833426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3d91-8b51-4967-b8fe-be0be33e9255 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:09.833721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3d91-8b51-4967-b8fe-be0be33e9255 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:09.834004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3d91-8b51-4967-b8fe-be0be33e9255 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:09.835366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 701/1412] 10.222.0.91 () {54 vars in 918 bytes} [Thu May 26 13:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:09.840297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191007-cecb-42a9-b56d-4f9d5fac9d63 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:09.840621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191007-cecb-42a9-b56d-4f9d5fac9d63 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:09.840904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191007-cecb-42a9-b56d-4f9d5fac9d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:09.862222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-94191007-cecb-42a9-b56d-4f9d5fac9d63 None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:09.863423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-94191007-cecb-42a9-b56d-4f9d5fac9d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:09.873024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 712/1413] 10.222.0.91 () {58 vars in 1015 bytes} [Thu May 26 13:49:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:12.588851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da6dda-f639-491b-aa33-5fbbe9591f85 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:12.589433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da6dda-f639-491b-aa33-5fbbe9591f85 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:12.589923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da6dda-f639-491b-aa33-5fbbe9591f85 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:12.591346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:12.597739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ec663-2962-482b-8218-881405c437dd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:12.598174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ec663-2962-482b-8218-881405c437dd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:12.598450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ec663-2962-482b-8218-881405c437dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:12.618040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bc6ec663-2962-482b-8218-881405c437dd None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:12.620083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bc6ec663-2962-482b-8218-881405c437dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:12.627577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:12.673637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da6dda-f639-491b-aa33-5fbbe9591f85 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:12.690638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da6dda-f639-491b-aa33-5fbbe9591f85 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:12.698500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6277c-189c-4a7f-b632-57eab11a354b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:12.699569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6277c-189c-4a7f-b632-57eab11a354b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:12.699891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6277c-189c-4a7f-b632-57eab11a354b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:12.700167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6277c-189c-4a7f-b632-57eab11a354b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:12.708920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6277c-189c-4a7f-b632-57eab11a354b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:12.710584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6277c-189c-4a7f-b632-57eab11a354b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:12.721051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 703/1416] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:49:13.733995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d3a5-7a39-4b24-8775-7c9a94f65603 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:13.734563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d3a5-7a39-4b24-8775-7c9a94f65603 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:13.734663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d3a5-7a39-4b24-8775-7c9a94f65603 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:13.736053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:13.742279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989c43c-7318-41a4-94ce-eeeff42bd0d2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:13.742736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989c43c-7318-41a4-94ce-eeeff42bd0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:13.742862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989c43c-7318-41a4-94ce-eeeff42bd0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:13.763339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f989c43c-7318-41a4-94ce-eeeff42bd0d2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:13.764592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f989c43c-7318-41a4-94ce-eeeff42bd0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:13.777130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:13.978142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1d3a5-7a39-4b24-8775-7c9a94f65603 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:13.993455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1d3a5-7a39-4b24-8775-7c9a94f65603 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:13.999473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc3b37-c903-4412-9c3f-043d9d1ea268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:14.000224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc3b37-c903-4412-9c3f-043d9d1ea268 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:14.000617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc3b37-c903-4412-9c3f-043d9d1ea268 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:14.000971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc3b37-c903-4412-9c3f-043d9d1ea268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:14.011093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc3b37-c903-4412-9c3f-043d9d1ea268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:14.012950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc3b37-c903-4412-9c3f-043d9d1ea268 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:14.023919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 715/1419] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:14.970166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f6cc-bcaa-41a5-84cc-51287e8c7429 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:14.970738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f6cc-bcaa-41a5-84cc-51287e8c7429 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:14.970895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f6cc-bcaa-41a5-84cc-51287e8c7429 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:14.972337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:14.978218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719e45-9903-4a29-9e62-80456315204a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:14.978902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719e45-9903-4a29-9e62-80456315204a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:14.978902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719e45-9903-4a29-9e62-80456315204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:14.997672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5a719e45-9903-4a29-9e62-80456315204a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:14.998880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5a719e45-9903-4a29-9e62-80456315204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:15.007640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:15.160979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519f6cc-bcaa-41a5-84cc-51287e8c7429 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:15.176085 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519f6cc-bcaa-41a5-84cc-51287e8c7429 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:15.182956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80972a-9740-4077-acd6-3fa4bb2ed92e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:15.183890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80972a-9740-4077-acd6-3fa4bb2ed92e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:15.184264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80972a-9740-4077-acd6-3fa4bb2ed92e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:15.184665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80972a-9740-4077-acd6-3fa4bb2ed92e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:15.195858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80972a-9740-4077-acd6-3fa4bb2ed92e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:15.197819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80972a-9740-4077-acd6-3fa4bb2ed92e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:15.207430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 706/1422] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:16.149754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff496a4-dede-46eb-bb14-706afcb9803c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:16.150338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff496a4-dede-46eb-bb14-706afcb9803c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:16.150338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff496a4-dede-46eb-bb14-706afcb9803c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:16.151892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:16.158087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984ede8-8c22-4ec8-9dfe-1df009913a51 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:16.158788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984ede8-8c22-4ec8-9dfe-1df009913a51 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:16.158982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984ede8-8c22-4ec8-9dfe-1df009913a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:16.187735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0984ede8-8c22-4ec8-9dfe-1df009913a51 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:16.188968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0984ede8-8c22-4ec8-9dfe-1df009913a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:16.199286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:16.354911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff496a4-dede-46eb-bb14-706afcb9803c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:16.378951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff496a4-dede-46eb-bb14-706afcb9803c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:16.387765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91b8e5-4522-443c-93e3-70014b38bd54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:16.388423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91b8e5-4522-443c-93e3-70014b38bd54 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:16.388706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91b8e5-4522-443c-93e3-70014b38bd54 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:16.388986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91b8e5-4522-443c-93e3-70014b38bd54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:16.399116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91b8e5-4522-443c-93e3-70014b38bd54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:16.400784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91b8e5-4522-443c-93e3-70014b38bd54 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:16.410782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 718/1425] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:17.386943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35ca9-118c-4b56-bf55-736f46b15046 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:17.387775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35ca9-118c-4b56-bf55-736f46b15046 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:17.387775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35ca9-118c-4b56-bf55-736f46b15046 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:17.388877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:49:17.394878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168892-29c0-4c6b-ac10-1b88f4e4f24b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:17.395556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168892-29c0-4c6b-ac10-1b88f4e4f24b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:17.395778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168892-29c0-4c6b-ac10-1b88f4e4f24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:17.417310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-00168892-29c0-4c6b-ac10-1b88f4e4f24b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:17.418542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-00168892-29c0-4c6b-ac10-1b88f4e4f24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:17.431133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:17.581836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35ca9-118c-4b56-bf55-736f46b15046 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:17.596263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35ca9-118c-4b56-bf55-736f46b15046 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:17.601965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066443b-52d5-4fa8-9752-4381b5d44df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:17.602614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066443b-52d5-4fa8-9752-4381b5d44df5 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:17.602898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066443b-52d5-4fa8-9752-4381b5d44df5 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:17.603179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066443b-52d5-4fa8-9752-4381b5d44df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:17.617613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066443b-52d5-4fa8-9752-4381b5d44df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:17.619311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066443b-52d5-4fa8-9752-4381b5d44df5 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:17.628858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 709/1428] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:18.485713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde845f2-90ef-4182-8cdf-293aac0694ab None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:18.486241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde845f2-90ef-4182-8cdf-293aac0694ab None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:18.486241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde845f2-90ef-4182-8cdf-293aac0694ab None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:18.487579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:18.493675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c496a2-3f9e-4626-b900-f6404046f35c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:18.494510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c496a2-3f9e-4626-b900-f6404046f35c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:18.494686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c496a2-3f9e-4626-b900-f6404046f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:18.515309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a3c496a2-3f9e-4626-b900-f6404046f35c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:18.517445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3c496a2-3f9e-4626-b900-f6404046f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:18.527952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:18.676657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde845f2-90ef-4182-8cdf-293aac0694ab None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:18.691341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde845f2-90ef-4182-8cdf-293aac0694ab None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:18.697570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fb397-6463-495b-b3ee-5dd0aaf628f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:18.698190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fb397-6463-495b-b3ee-5dd0aaf628f0 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:18.698475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fb397-6463-495b-b3ee-5dd0aaf628f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:18.698754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fb397-6463-495b-b3ee-5dd0aaf628f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:18.709555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fb397-6463-495b-b3ee-5dd0aaf628f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:18.711626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fb397-6463-495b-b3ee-5dd0aaf628f0 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:18.722794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 721/1431] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:49:19.591786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9fbda-9e39-4114-b02f-c0cfd44d3815 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.592395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9fbda-9e39-4114-b02f-c0cfd44d3815 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.592395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9fbda-9e39-4114-b02f-c0cfd44d3815 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.593980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:19.599467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63566c2-137d-4bac-b351-c54e507ec020 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.601064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63566c2-137d-4bac-b351-c54e507ec020 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.601402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63566c2-137d-4bac-b351-c54e507ec020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.622504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b63566c2-137d-4bac-b351-c54e507ec020 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:19.623756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b63566c2-137d-4bac-b351-c54e507ec020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:19.639130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:19.797517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9fbda-9e39-4114-b02f-c0cfd44d3815 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:19.816009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9fbda-9e39-4114-b02f-c0cfd44d3815 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:19.821603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96cf5b-9d99-476f-bdb2-168225a402cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:19.822245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cf5b-9d99-476f-bdb2-168225a402cf service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.822526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cf5b-9d99-476f-bdb2-168225a402cf service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.822799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cf5b-9d99-476f-bdb2-168225a402cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.831486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96cf5b-9d99-476f-bdb2-168225a402cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:19.833193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96cf5b-9d99-476f-bdb2-168225a402cf service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:19.844177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 712/1434] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:20.729660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca17fd9-489d-4d29-84cd-6d9751050f13 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:20.730263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca17fd9-489d-4d29-84cd-6d9751050f13 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:20.730263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca17fd9-489d-4d29-84cd-6d9751050f13 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:20.731571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:49:20.737510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9468-6dde-4b68-a37f-d478edcb76d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:20.737994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9468-6dde-4b68-a37f-d478edcb76d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:20.738377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9468-6dde-4b68-a37f-d478edcb76d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:20.758731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2a9a9468-6dde-4b68-a37f-d478edcb76d0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:20.760761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a9a9468-6dde-4b68-a37f-d478edcb76d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:20.769199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:20.920114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca17fd9-489d-4d29-84cd-6d9751050f13 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:20.935477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca17fd9-489d-4d29-84cd-6d9751050f13 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:20.941196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759733b-7706-43fa-b842-1b5b6e7a7273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:20.941846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759733b-7706-43fa-b842-1b5b6e7a7273 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:20.942138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759733b-7706-43fa-b842-1b5b6e7a7273 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:20.942417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759733b-7706-43fa-b842-1b5b6e7a7273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:20.952034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759733b-7706-43fa-b842-1b5b6e7a7273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:20.953709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759733b-7706-43fa-b842-1b5b6e7a7273 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:20.964250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 724/1437] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:21.840972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d199-0eab-4f04-a218-652212f1a6ed None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:21.841604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d199-0eab-4f04-a218-652212f1a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:21.841604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d199-0eab-4f04-a218-652212f1a6ed None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:21.842969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:49:21.850216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1121a6-6d0e-4900-bf16-641c5f0eb57a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:21.850728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1121a6-6d0e-4900-bf16-641c5f0eb57a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:21.850908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1121a6-6d0e-4900-bf16-641c5f0eb57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:21.870601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5b1121a6-6d0e-4900-bf16-641c5f0eb57a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:21.871735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5b1121a6-6d0e-4900-bf16-641c5f0eb57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:21.879900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:22.032732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27d199-0eab-4f04-a218-652212f1a6ed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:22.048607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27d199-0eab-4f04-a218-652212f1a6ed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:22.055327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fb0bb-94e7-4d3d-aca8-c4b040a79384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:22.055974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fb0bb-94e7-4d3d-aca8-c4b040a79384 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:22.056254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fb0bb-94e7-4d3d-aca8-c4b040a79384 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:22.056597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fb0bb-94e7-4d3d-aca8-c4b040a79384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:22.065168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fb0bb-94e7-4d3d-aca8-c4b040a79384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:22.066776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fb0bb-94e7-4d3d-aca8-c4b040a79384 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:22.076967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 715/1440] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:24.185136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e423ce0-fa3f-4a40-894b-ed8450ec1644 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:24.185755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e423ce0-fa3f-4a40-894b-ed8450ec1644 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:24.185755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e423ce0-fa3f-4a40-894b-ed8450ec1644 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:24.187229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:24.193098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18075a7-d385-4cdf-a7e9-9ecda245ffc1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:24.193977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18075a7-d385-4cdf-a7e9-9ecda245ffc1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:24.193977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18075a7-d385-4cdf-a7e9-9ecda245ffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:24.214582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c18075a7-d385-4cdf-a7e9-9ecda245ffc1 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:24.215861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c18075a7-d385-4cdf-a7e9-9ecda245ffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:24.224229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:24.263497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e423ce0-fa3f-4a40-894b-ed8450ec1644 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:24.275148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e423ce0-fa3f-4a40-894b-ed8450ec1644 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:24.278793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ac8e6-faf5-4138-8a81-f7140d5e8d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:24.279438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ac8e6-faf5-4138-8a81-f7140d5e8d2b service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:24.279723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ac8e6-faf5-4138-8a81-f7140d5e8d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:24.280000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ac8e6-faf5-4138-8a81-f7140d5e8d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:24.291622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ac8e6-faf5-4138-8a81-f7140d5e8d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:24.293243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ac8e6-faf5-4138-8a81-f7140d5e8d2b service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:24.303333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 727/1443] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:26.735033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36203575-ea53-4b65-8e09-222fd6fa933b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:26.735695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36203575-ea53-4b65-8e09-222fd6fa933b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:26.735695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36203575-ea53-4b65-8e09-222fd6fa933b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:26.736973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:26.744088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e0ca3-d054-4250-8afe-6950217256b4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:26.744595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e0ca3-d054-4250-8afe-6950217256b4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:26.744849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e0ca3-d054-4250-8afe-6950217256b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:26.768194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-631e0ca3-d054-4250-8afe-6950217256b4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:26.769961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-631e0ca3-d054-4250-8afe-6950217256b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:26.779741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:26.821356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36203575-ea53-4b65-8e09-222fd6fa933b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:26.823282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36203575-ea53-4b65-8e09-222fd6fa933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:26.845699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36203575-ea53-4b65-8e09-222fd6fa933b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:26.850131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da2a58-9947-444d-9072-012f1ef0306f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:26.850787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da2a58-9947-444d-9072-012f1ef0306f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:26.851160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da2a58-9947-444d-9072-012f1ef0306f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:26.851442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da2a58-9947-444d-9072-012f1ef0306f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:26.861987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da2a58-9947-444d-9072-012f1ef0306f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:26.865859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da2a58-9947-444d-9072-012f1ef0306f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:26.876400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 718/1446] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:29.517952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94e83a-c5b4-45b9-aee5-9e151156a819 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:29.518848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94e83a-c5b4-45b9-aee5-9e151156a819 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:29.519885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94e83a-c5b4-45b9-aee5-9e151156a819 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:29.524842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:29.530915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed107b58-9a6a-45d5-847c-12c03285b06a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:29.533456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed107b58-9a6a-45d5-847c-12c03285b06a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:29.533456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed107b58-9a6a-45d5-847c-12c03285b06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:29.550751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ed107b58-9a6a-45d5-847c-12c03285b06a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:29.551955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ed107b58-9a6a-45d5-847c-12c03285b06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:29.560012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:29.602174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94e83a-c5b4-45b9-aee5-9e151156a819 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:29.613621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94e83a-c5b4-45b9-aee5-9e151156a819 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:29.618849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b823b-9c1d-4ca3-b87e-1aa9c7ca2b2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:29.619507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b823b-9c1d-4ca3-b87e-1aa9c7ca2b2d service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:29.619790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b823b-9c1d-4ca3-b87e-1aa9c7ca2b2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:29.620070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b823b-9c1d-4ca3-b87e-1aa9c7ca2b2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:29.630441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b823b-9c1d-4ca3-b87e-1aa9c7ca2b2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:29.631895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b823b-9c1d-4ca3-b87e-1aa9c7ca2b2d service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:29.642015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 730/1449] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:49:32.059893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3c2b-3683-4d61-83a7-7e2c0f48214c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:32.060493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3c2b-3683-4d61-83a7-7e2c0f48214c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:32.060493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3c2b-3683-4d61-83a7-7e2c0f48214c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:32.061873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:32.068474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7afe1-dead-4d17-9c56-2993f8196742 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:32.069105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7afe1-dead-4d17-9c56-2993f8196742 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:32.069105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7afe1-dead-4d17-9c56-2993f8196742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:32.102068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-90d7afe1-dead-4d17-9c56-2993f8196742 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:32.103400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-90d7afe1-dead-4d17-9c56-2993f8196742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:32.112237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:32.153254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b3c2b-3683-4d61-83a7-7e2c0f48214c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:32.166404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b3c2b-3683-4d61-83a7-7e2c0f48214c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:32.170441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb5ff-eda3-4983-9d91-2a0485e2e10e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:32.171119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb5ff-eda3-4983-9d91-2a0485e2e10e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:32.171410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb5ff-eda3-4983-9d91-2a0485e2e10e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:32.171683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb5ff-eda3-4983-9d91-2a0485e2e10e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:32.181592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb5ff-eda3-4983-9d91-2a0485e2e10e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:32.183050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb5ff-eda3-4983-9d91-2a0485e2e10e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:32.193684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 721/1452] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:34.503102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280de5-ebb7-493b-a754-702fbd08dfd7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:34.503815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280de5-ebb7-493b-a754-702fbd08dfd7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:34.503815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280de5-ebb7-493b-a754-702fbd08dfd7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:34.506375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:34.514052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340669-70ba-465e-8a8b-d3992e86b973 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:34.514614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340669-70ba-465e-8a8b-d3992e86b973 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:34.514614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340669-70ba-465e-8a8b-d3992e86b973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:34.534029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a2340669-70ba-465e-8a8b-d3992e86b973 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:34.535269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a2340669-70ba-465e-8a8b-d3992e86b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:34.543833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:34.584166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37280de5-ebb7-493b-a754-702fbd08dfd7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:34.595633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37280de5-ebb7-493b-a754-702fbd08dfd7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:34.599314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d70588-b507-49ef-b4e7-cc9791e2d72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:34.599996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d70588-b507-49ef-b4e7-cc9791e2d72d service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:34.600278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d70588-b507-49ef-b4e7-cc9791e2d72d service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:34.600554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d70588-b507-49ef-b4e7-cc9791e2d72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:34.610475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d70588-b507-49ef-b4e7-cc9791e2d72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:34.611922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d70588-b507-49ef-b4e7-cc9791e2d72d service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:34.622021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 733/1455] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:37.016613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1759d-3dad-4a0f-bc77-f1e59f07eb24 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:37.017200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1759d-3dad-4a0f-bc77-f1e59f07eb24 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:37.017200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1759d-3dad-4a0f-bc77-f1e59f07eb24 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:37.018708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:37.024299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801ccad-3c98-4740-8027-07a887d163f2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:37.024796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801ccad-3c98-4740-8027-07a887d163f2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:37.024879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801ccad-3c98-4740-8027-07a887d163f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:37.031408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=70 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:37.045046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4801ccad-3c98-4740-8027-07a887d163f2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:37.046443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4801ccad-3c98-4740-8027-07a887d163f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:37.055108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:37.094463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1759d-3dad-4a0f-bc77-f1e59f07eb24 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:37.099929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=64 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:37.108512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1759d-3dad-4a0f-bc77-f1e59f07eb24 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:37.112233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f6db5-c3de-48ed-84d9-5c42c47b261f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:37.112912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f6db5-c3de-48ed-84d9-5c42c47b261f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:37.113228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f6db5-c3de-48ed-84d9-5c42c47b261f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:37.113536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f6db5-c3de-48ed-84d9-5c42c47b261f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:37.123691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f6db5-c3de-48ed-84d9-5c42c47b261f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:37.125160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f6db5-c3de-48ed-84d9-5c42c47b261f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:37.135732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 724/1458] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:39.545356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4877b4-b460-4918-a03c-1d0c2d6e4d19 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:39.545933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4877b4-b460-4918-a03c-1d0c2d6e4d19 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:39.545933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4877b4-b460-4918-a03c-1d0c2d6e4d19 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:39.547319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:39.553879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f662476-f332-4c14-b5fa-d0020333bebd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:39.554466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f662476-f332-4c14-b5fa-d0020333bebd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:39.554466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f662476-f332-4c14-b5fa-d0020333bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:39.573679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f662476-f332-4c14-b5fa-d0020333bebd None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:39.574867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f662476-f332-4c14-b5fa-d0020333bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:39.583055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:39.621226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4877b4-b460-4918-a03c-1d0c2d6e4d19 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:39.634306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4877b4-b460-4918-a03c-1d0c2d6e4d19 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:39.638505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198e260-fe0b-4672-a35a-1c9a29b813d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:39.639203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e260-fe0b-4672-a35a-1c9a29b813d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:39.639597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e260-fe0b-4672-a35a-1c9a29b813d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:39.639887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e260-fe0b-4672-a35a-1c9a29b813d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:39.650948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198e260-fe0b-4672-a35a-1c9a29b813d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:39.652497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198e260-fe0b-4672-a35a-1c9a29b813d4 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:39.663621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 736/1461] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:41.950053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07f580-1306-4f1b-9981-c58cd848f844 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:41.950726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07f580-1306-4f1b-9981-c58cd848f844 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:41.950726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07f580-1306-4f1b-9981-c58cd848f844 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:41.952045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:41.957658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47e60e-d9e2-423b-ba53-4b33aa0b9d62 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:41.958880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47e60e-d9e2-423b-ba53-4b33aa0b9d62 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:41.958880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47e60e-d9e2-423b-ba53-4b33aa0b9d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:41.977502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9b47e60e-d9e2-423b-ba53-4b33aa0b9d62 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:41.978647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9b47e60e-d9e2-423b-ba53-4b33aa0b9d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:41.986934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:42.025075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07f580-1306-4f1b-9981-c58cd848f844 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:42.035960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07f580-1306-4f1b-9981-c58cd848f844 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:42.039811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9c19d-9cfe-4f8e-8f6f-621b781d12d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:42.040449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c19d-9cfe-4f8e-8f6f-621b781d12d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:42.040735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c19d-9cfe-4f8e-8f6f-621b781d12d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:42.041011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c19d-9cfe-4f8e-8f6f-621b781d12d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:42.050205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c19d-9cfe-4f8e-8f6f-621b781d12d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:42.052331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c19d-9cfe-4f8e-8f6f-621b781d12d1 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:42.061904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 727/1464] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:44.401756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffda6f-6b64-4a8c-9ba3-ae2cfb7df320 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:44.402391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffda6f-6b64-4a8c-9ba3-ae2cfb7df320 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:44.402391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffda6f-6b64-4a8c-9ba3-ae2cfb7df320 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:44.403680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:44.409108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4875a-29e2-489e-81c7-d84a1ff6dc00 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:44.409607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4875a-29e2-489e-81c7-d84a1ff6dc00 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:44.409709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4875a-29e2-489e-81c7-d84a1ff6dc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:44.428034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f0c4875a-29e2-489e-81c7-d84a1ff6dc00 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:44.429266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f0c4875a-29e2-489e-81c7-d84a1ff6dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:44.437049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:44.473554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffda6f-6b64-4a8c-9ba3-ae2cfb7df320 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:44.484735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffda6f-6b64-4a8c-9ba3-ae2cfb7df320 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:44.488290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b030b-4840-4297-a41a-4de0464207bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:44.488934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b030b-4840-4297-a41a-4de0464207bd service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:44.489305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b030b-4840-4297-a41a-4de0464207bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:44.489599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b030b-4840-4297-a41a-4de0464207bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:44.498794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b030b-4840-4297-a41a-4de0464207bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:44.500230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b030b-4840-4297-a41a-4de0464207bd service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:44.509734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 739/1467] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:46.782539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722af17e-4cf6-4918-b84c-3c65ccda852c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:46.783005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722af17e-4cf6-4918-b84c-3c65ccda852c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:46.783109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722af17e-4cf6-4918-b84c-3c65ccda852c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:46.784476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:46.792642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5219-0a8d-4bcb-9e9d-64c5a7b7ae2f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:46.793275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5219-0a8d-4bcb-9e9d-64c5a7b7ae2f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:46.793275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5219-0a8d-4bcb-9e9d-64c5a7b7ae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:46.814192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-26ed5219-0a8d-4bcb-9e9d-64c5a7b7ae2f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:46.815417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-26ed5219-0a8d-4bcb-9e9d-64c5a7b7ae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:46.823412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:46.861363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722af17e-4cf6-4918-b84c-3c65ccda852c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:46.873019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722af17e-4cf6-4918-b84c-3c65ccda852c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:46.877393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b556e29-cba8-46cc-8d27-4bdf31c7ad47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:46.878076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b556e29-cba8-46cc-8d27-4bdf31c7ad47 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:46.878360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b556e29-cba8-46cc-8d27-4bdf31c7ad47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:46.878638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b556e29-cba8-46cc-8d27-4bdf31c7ad47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:46.888804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b556e29-cba8-46cc-8d27-4bdf31c7ad47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:46.890292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b556e29-cba8-46cc-8d27-4bdf31c7ad47 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:46.901133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 730/1470] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:49:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:56.808068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=20 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:56.895959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=23 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:50:20.293874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956fd2-c125-420f-96ec-2f6e23006526 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:20.294472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956fd2-c125-420f-96ec-2f6e23006526 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:20.294472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956fd2-c125-420f-96ec-2f6e23006526 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:20.295810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 741/1471] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:20.301352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a470d-ca72-4918-9f92-81b523ee0c99 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:20.301799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a470d-ca72-4918-9f92-81b523ee0c99 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:20.301908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a470d-ca72-4918-9f92-81b523ee0c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:20.320512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b18a470d-ca72-4918-9f92-81b523ee0c99 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:20.321730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b18a470d-ca72-4918-9f92-81b523ee0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:20.329332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 731/1472] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:20.375807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01cf43-f708-4fb6-851f-88ba1fce30d0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:20.376106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01cf43-f708-4fb6-851f-88ba1fce30d0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:20.376387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01cf43-f708-4fb6-851f-88ba1fce30d0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:20.377853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 742/1473] 10.222.0.91 () {54 vars in 962 bytes} [Thu May 26 13:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:20.383679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058d0a5-6a48-4dc9-9277-d8430a372eac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:20.383963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058d0a5-6a48-4dc9-9277-d8430a372eac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:20.384236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058d0a5-6a48-4dc9-9277-d8430a372eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:20.401688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c058d0a5-6a48-4dc9-9277-d8430a372eac None None] MFA Rules not processed for user `a0c209755dfc4b4089176a056cf2f683`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:20.402863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c058d0a5-6a48-4dc9-9277-d8430a372eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:20.409087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 732/1474] 10.222.0.91 () {58 vars in 1059 bytes} [Thu May 26 13:50:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:20.416331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01cf43-f708-4fb6-851f-88ba1fce30d0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:20.427447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01cf43-f708-4fb6-851f-88ba1fce30d0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:20.431066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5fab-3e66-45e5-b139-5821f34cea7b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:20.431697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5fab-3e66-45e5-b139-5821f34cea7b service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:20.431987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5fab-3e66-45e5-b139-5821f34cea7b service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:20.432270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5fab-3e66-45e5-b139-5821f34cea7b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:20.441858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a5fab-3e66-45e5-b139-5821f34cea7b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:20.443434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a5fab-3e66-45e5-b139-5821f34cea7b service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:20.452825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 743/1475] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:22.720216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23321b5a-e5c6-4d39-bba2-6ceae517acd5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:22.720797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23321b5a-e5c6-4d39-bba2-6ceae517acd5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:22.720797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23321b5a-e5c6-4d39-bba2-6ceae517acd5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:22.722205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 733/1476] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:22.727304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a3af-7979-4b3d-a2f1-db491412d6cb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:22.727766 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a3af-7979-4b3d-a2f1-db491412d6cb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:22.727876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a3af-7979-4b3d-a2f1-db491412d6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:22.746046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-00f7a3af-7979-4b3d-a2f1-db491412d6cb None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:22.746922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-00f7a3af-7979-4b3d-a2f1-db491412d6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:22.755334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 744/1477] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:22.777701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23321b5a-e5c6-4d39-bba2-6ceae517acd5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:22.790146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23321b5a-e5c6-4d39-bba2-6ceae517acd5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:22.794532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410d2cb-99eb-4f3e-a35c-d0e1cc261d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:22.795169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410d2cb-99eb-4f3e-a35c-d0e1cc261d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:22.795444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410d2cb-99eb-4f3e-a35c-d0e1cc261d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:22.795720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410d2cb-99eb-4f3e-a35c-d0e1cc261d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:22.796442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410d2cb-99eb-4f3e-a35c-d0e1cc261d4d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:22.798254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-f410d2cb-99eb-4f3e-a35c-d0e1cc261d4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:50:22.798254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:50:22.798254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:50:22.799504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410d2cb-99eb-4f3e-a35c-d0e1cc261d4d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:22.819464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 734/1478] 10.222.0.91 () {56 vars in 1108 bytes} [Thu May 26 13:50:22 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:50:25.030980 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290c85-bef6-4e21-9b7d-7c05c3e3d111 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:25.031547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290c85-bef6-4e21-9b7d-7c05c3e3d111 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:25.031547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290c85-bef6-4e21-9b7d-7c05c3e3d111 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:25.032950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 745/1479] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:25.038485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a32b98-1565-4b25-a2d1-5ce03788999f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:25.038916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a32b98-1565-4b25-a2d1-5ce03788999f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:25.039044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a32b98-1565-4b25-a2d1-5ce03788999f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:25.057129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-15a32b98-1565-4b25-a2d1-5ce03788999f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:25.058280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15a32b98-1565-4b25-a2d1-5ce03788999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:25.066509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 735/1480] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:25.109653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48290c85-bef6-4e21-9b7d-7c05c3e3d111 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:25.124200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48290c85-bef6-4e21-9b7d-7c05c3e3d111 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:25.130343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da5242-b83e-43ae-8ab2-a4f74c1e352f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:25.131118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da5242-b83e-43ae-8ab2-a4f74c1e352f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:25.131427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da5242-b83e-43ae-8ab2-a4f74c1e352f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:25.131703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da5242-b83e-43ae-8ab2-a4f74c1e352f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:25.139865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da5242-b83e-43ae-8ab2-a4f74c1e352f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:25.141490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da5242-b83e-43ae-8ab2-a4f74c1e352f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:25.150827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 746/1481] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:50:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:27.527672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109af84e-a569-417b-9dac-e7326613f601 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:27.528190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109af84e-a569-417b-9dac-e7326613f601 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:27.528190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109af84e-a569-417b-9dac-e7326613f601 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:27.529640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 736/1482] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:27.535311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a453-c43b-4846-986e-e9ce62253436 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:27.535858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a453-c43b-4846-986e-e9ce62253436 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:27.535858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a453-c43b-4846-986e-e9ce62253436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:27.553930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-aca6a453-c43b-4846-986e-e9ce62253436 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:27.555106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aca6a453-c43b-4846-986e-e9ce62253436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:27.562983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 747/1483] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:27.601764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109af84e-a569-417b-9dac-e7326613f601 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:27.612363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109af84e-a569-417b-9dac-e7326613f601 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:27.616038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c182e38-2710-4c41-9e8b-e35532004bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:27.616674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c182e38-2710-4c41-9e8b-e35532004bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:27.616957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c182e38-2710-4c41-9e8b-e35532004bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:27.617272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c182e38-2710-4c41-9e8b-e35532004bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:27.627863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c182e38-2710-4c41-9e8b-e35532004bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:27.629835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c182e38-2710-4c41-9e8b-e35532004bd5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:27.639404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 737/1484] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:50:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:29.885694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572f5e-22b6-4b8a-bc10-f59129c504c0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:29.886274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572f5e-22b6-4b8a-bc10-f59129c504c0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:29.886274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572f5e-22b6-4b8a-bc10-f59129c504c0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:29.887632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 748/1485] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:29.892869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ab1a1-b2e1-47b3-b2a6-f54a859480f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:29.893307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ab1a1-b2e1-47b3-b2a6-f54a859480f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:29.893467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ab1a1-b2e1-47b3-b2a6-f54a859480f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:29.912172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a02ab1a1-b2e1-47b3-b2a6-f54a859480f6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:29.913415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a02ab1a1-b2e1-47b3-b2a6-f54a859480f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:29.926489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 738/1486] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:29.966797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572f5e-22b6-4b8a-bc10-f59129c504c0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:29.977410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572f5e-22b6-4b8a-bc10-f59129c504c0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:29.981083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f1306-da1b-4aed-80ac-b0ed6cfc4b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:29.981750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f1306-da1b-4aed-80ac-b0ed6cfc4b23 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:29.982032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f1306-da1b-4aed-80ac-b0ed6cfc4b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:29.982310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f1306-da1b-4aed-80ac-b0ed6cfc4b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:29.991142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f1306-da1b-4aed-80ac-b0ed6cfc4b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:29.992573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f1306-da1b-4aed-80ac-b0ed6cfc4b23 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:30.001959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 749/1487] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:32.314201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc417d-cd7b-4e11-9ebc-657719401da2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:32.314800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc417d-cd7b-4e11-9ebc-657719401da2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:32.314800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc417d-cd7b-4e11-9ebc-657719401da2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:32.316191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 739/1488] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:32.321505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c8cee-6cc6-4f66-a78e-913df4c8b3d6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:32.321943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c8cee-6cc6-4f66-a78e-913df4c8b3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:32.322071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c8cee-6cc6-4f66-a78e-913df4c8b3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:32.339956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-274c8cee-6cc6-4f66-a78e-913df4c8b3d6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:32.341170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-274c8cee-6cc6-4f66-a78e-913df4c8b3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:32.349485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 750/1489] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:32.388687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc417d-cd7b-4e11-9ebc-657719401da2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:32.400098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc417d-cd7b-4e11-9ebc-657719401da2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:32.403501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec2650-cbb0-458a-9e46-29413b9a0a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:32.404138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec2650-cbb0-458a-9e46-29413b9a0a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:32.404419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec2650-cbb0-458a-9e46-29413b9a0a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:32.404691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec2650-cbb0-458a-9e46-29413b9a0a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:32.414653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec2650-cbb0-458a-9e46-29413b9a0a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:32.416074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec2650-cbb0-458a-9e46-29413b9a0a1d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:32.425271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 740/1490] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:34.833646 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5a25-ea21-436f-8170-fe3f35b8c4d3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:34.834150 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5a25-ea21-436f-8170-fe3f35b8c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:34.834263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5a25-ea21-436f-8170-fe3f35b8c4d3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:34.835583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 751/1491] 10.222.0.91 () {54 vars in 926 bytes} [Thu May 26 13:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:50:34.840710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eac57-ada7-40d1-b05a-a627bff09292 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:34.841110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eac57-ada7-40d1-b05a-a627bff09292 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:34.841299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eac57-ada7-40d1-b05a-a627bff09292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:34.859272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5c5eac57-ada7-40d1-b05a-a627bff09292 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:34.860464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c5eac57-ada7-40d1-b05a-a627bff09292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:34.868112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 741/1492] 10.222.0.91 () {58 vars in 1023 bytes} [Thu May 26 13:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:34.906108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a5a25-ea21-436f-8170-fe3f35b8c4d3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:34.917718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a5a25-ea21-436f-8170-fe3f35b8c4d3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:34.921252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3839a-2f27-47a9-aa29-9e84be49a096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:34.921888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3839a-2f27-47a9-aa29-9e84be49a096 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:34.922176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3839a-2f27-47a9-aa29-9e84be49a096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:34.922453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3839a-2f27-47a9-aa29-9e84be49a096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:34.931378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3839a-2f27-47a9-aa29-9e84be49a096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:34.932902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3839a-2f27-47a9-aa29-9e84be49a096 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:34.942778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 752/1493] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:44.854126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=33 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:50:44.938168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=33 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:15.443092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7f8ca-d704-48ef-a7b8-97f7294f4a32 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.443687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7f8ca-d704-48ef-a7b8-97f7294f4a32 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.443687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7f8ca-d704-48ef-a7b8-97f7294f4a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.461834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a5f7f8ca-d704-48ef-a7b8-97f7294f4a32 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:15.462965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5f7f8ca-d704-48ef-a7b8-97f7294f4a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:15.471246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 742/1494] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:15.480313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b62405-d86f-4a0d-bf09-449b8026326a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.480823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b62405-d86f-4a0d-bf09-449b8026326a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.480921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b62405-d86f-4a0d-bf09-449b8026326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.499151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a3b62405-d86f-4a0d-bf09-449b8026326a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:15.500335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a3b62405-d86f-4a0d-bf09-449b8026326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:15.508366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 753/1495] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:15.512795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7f8ca-d704-48ef-a7b8-97f7294f4a32 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.522362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7f8ca-d704-48ef-a7b8-97f7294f4a32 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.526870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.527522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.527822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.528089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.528812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:52:15.529120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.530700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:15.530700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:52:15.530700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:52:15.531579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.545014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 743/1496] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:52:15.553684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b62405-d86f-4a0d-bf09-449b8026326a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.562931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b62405-d86f-4a0d-bf09-449b8026326a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.567502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.568153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.568439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.568709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.569433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.571229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:15.571229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) May 26 13:52:15.571229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: May 26 13:52:15.572424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.612950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 754/1497] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:52:15 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:15.615745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.626080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b5dab-8515-4781-becd-5c2f20a23988 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.682126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.682797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.683082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.683376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.684063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.685074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:15.685074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:52:15.685074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:52:15.686323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.706911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 744/1498] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:52:15 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:15.711054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.721666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9a7be-4412-44c7-935f-01b21dc4089e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.726045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.726695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.726975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.727244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.727994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.729034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:15.729034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) May 26 13:52:15.729034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: May 26 13:52:15.729948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.743000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 755/1499] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:15.747046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.757500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03985134-dfed-408e-b5a9-1654f842a70d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.761949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9735487-6485-48ab-8d63-392ba98fc8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.762600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9735487-6485-48ab-8d63-392ba98fc8bb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.762878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9735487-6485-48ab-8d63-392ba98fc8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.763149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9735487-6485-48ab-8d63-392ba98fc8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.763859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9735487-6485-48ab-8d63-392ba98fc8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.764894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-b9735487-6485-48ab-8d63-392ba98fc8bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:15.764894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:52:15.764894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:52:15.765805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9735487-6485-48ab-8d63-392ba98fc8bb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.779037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 745/1500] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:15.783723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.793708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370ece8-4bfd-443b-a9ae-6efadd950be4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.798891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a227fd5-2e36-44be-a58f-4b24d774bc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.799763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a227fd5-2e36-44be-a58f-4b24d774bc8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.800047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a227fd5-2e36-44be-a58f-4b24d774bc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.800323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a227fd5-2e36-44be-a58f-4b24d774bc8d admin admin] PATH_INFO: `/v3/projects/2b418cdc24c54386846c85632d63ca41/users/b9949f1d36514142919c66cbe0d5d0ce/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.814862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a227fd5-2e36-44be-a58f-4b24d774bc8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b418cdc24c54386846c85632d63ca41, user_id=b9949f1d36514142919c66cbe0d5d0ce, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.815900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-6a227fd5-2e36-44be-a58f-4b24d774bc8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:15.815900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) May 26 13:52:15.815900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: May 26 13:52:15.817873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a227fd5-2e36-44be-a58f-4b24d774bc8d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.830803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 756/1501] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:52:15 2022] PUT /identity/v3/projects/2b418cdc24c54386846c85632d63ca41/users/b9949f1d36514142919c66cbe0d5d0ce/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:15.841812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeda04-2554-4395-bd39-10beffe132c8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.842203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeda04-2554-4395-bd39-10beffe132c8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.842476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeda04-2554-4395-bd39-10beffe132c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.868650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bfeeda04-2554-4395-bd39-10beffe132c8 None None] MFA Rules not processed for user `b9949f1d36514142919c66cbe0d5d0ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:15.869880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfeeda04-2554-4395-bd39-10beffe132c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:15.933852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 746/1502] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:52:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:15.946794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5c498-fae8-4758-83e0-6146a32fd44e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.947091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5c498-fae8-4758-83e0-6146a32fd44e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.947373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5c498-fae8-4758-83e0-6146a32fd44e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.948728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 757/1503] 10.222.0.91 () {54 vars in 963 bytes} [Thu May 26 13:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:52:15.953919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd67991-c7df-48e9-90c3-be16e91873ad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.954208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd67991-c7df-48e9-90c3-be16e91873ad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.954484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd67991-c7df-48e9-90c3-be16e91873ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.972478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dcd67991-c7df-48e9-90c3-be16e91873ad None None] MFA Rules not processed for user `8a381dccf1ca4dacaab4e0ae0d473525`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:15.973639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dcd67991-c7df-48e9-90c3-be16e91873ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.018718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:16.027276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 747/1504] 10.222.0.91 () {58 vars in 1060 bytes} [Thu May 26 13:52:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:16.036039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5c498-fae8-4758-83e0-6146a32fd44e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.047142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5c498-fae8-4758-83e0-6146a32fd44e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.051566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.052328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.052698 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.052977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.062152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.063590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.076200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 758/1505] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:52:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:16.120781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236ae4-ee5a-4915-ad43-f0b7a6996681 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.121073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236ae4-ee5a-4915-ad43-f0b7a6996681 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.121377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236ae4-ee5a-4915-ad43-f0b7a6996681 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.122658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 748/1506] 10.222.0.91 () {52 vars in 829 bytes} [Thu May 26 13:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:52:16.126235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.188282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95454c-dd9f-437b-be8e-df5ac6fa4d0e service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.193102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.194017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.194393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.195412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] PATH_INFO: `/v3/users/b9949f1d36514142919c66cbe0d5d0ce` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.196311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9949f1d36514142919c66cbe0d5d0ce)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.197921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:16.197921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) May 26 13:52:16.197921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: May 26 13:52:16.199912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.273721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9949f1d36514142919c66cbe0d5d0ce'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.292486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9949f1d36514142919c66cbe0d5d0ce'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.293266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9949f1d36514142919c66cbe0d5d0ce'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.293982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9949f1d36514142919c66cbe0d5d0ce'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.302957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9949f1d36514142919c66cbe0d5d0ce'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.312630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:16.324785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 759/1507] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:52:16 2022] DELETE /identity/v3/users/b9949f1d36514142919c66cbe0d5d0ce => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:16.329931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.330220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.330547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.348068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:16.349308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.407536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 749/1508] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:16.417641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.419156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.478729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b16e8-7594-473a-bcdf-66dc1dc5e3c5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.482267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb32fe-42f0-4603-9a9f-c2482ef9b80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.482914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb32fe-42f0-4603-9a9f-c2482ef9b80b service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.483229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb32fe-42f0-4603-9a9f-c2482ef9b80b service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.483506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb32fe-42f0-4603-9a9f-c2482ef9b80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.493675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb32fe-42f0-4603-9a9f-c2482ef9b80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.495438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb32fe-42f0-4603-9a9f-c2482ef9b80b service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.505517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 760/1509] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:16.874808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.876533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.888676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050bae0-581e-4081-a265-90e6c370b419 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.895756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.896655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.896945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.901182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] PATH_INFO: `/v3/projects/2b418cdc24c54386846c85632d63ca41` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.901182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b418cdc24c54386846c85632d63ca41)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.901182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:52:16.901182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:52:16.901182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:52:16.901182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.963376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b418cdc24c54386846c85632d63ca41', 'request_id': 'req-78dbd75f-e6fb-43d9-b75e-6d27539a7859'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.972878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b418cdc24c54386846c85632d63ca41', 'request_id': 'req-78dbd75f-e6fb-43d9-b75e-6d27539a7859'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.988034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b418cdc24c54386846c85632d63ca41', 'request_id': 'req-78dbd75f-e6fb-43d9-b75e-6d27539a7859'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.999742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] The token cache is being invalidate because project 2b418cdc24c54386846c85632d63ca41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:17.000027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:17.000993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78dbd75f-e6fb-43d9-b75e-6d27539a7859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:17.002629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 750/1510] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:52:16 2022] DELETE /identity/v3/projects/2b418cdc24c54386846c85632d63ca41 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:26.501350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=29 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:26.996068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:59.023148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbea71-0879-46c3-930a-cbaed16c3120 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:59.023716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbea71-0879-46c3-930a-cbaed16c3120 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:59.023716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbea71-0879-46c3-930a-cbaed16c3120 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:59.025121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:53:59.029898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51463079-03e2-4082-987d-2e2edc13b675 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:59.030330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51463079-03e2-4082-987d-2e2edc13b675 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:59.030463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51463079-03e2-4082-987d-2e2edc13b675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:59.048945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-51463079-03e2-4082-987d-2e2edc13b675 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:59.050200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-51463079-03e2-4082-987d-2e2edc13b675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:59.108540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:59.278347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbea71-0879-46c3-930a-cbaed16c3120 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:59.280095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68dbea71-0879-46c3-930a-cbaed16c3120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:59.354072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbea71-0879-46c3-930a-cbaed16c3120 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:59.360126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d091d7-cc56-4b92-a500-a61b83bd2c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:59.360780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d091d7-cc56-4b92-a500-a61b83bd2c5e service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:59.361068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d091d7-cc56-4b92-a500-a61b83bd2c5e service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:59.361377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d091d7-cc56-4b92-a500-a61b83bd2c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:59.370095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d091d7-cc56-4b92-a500-a61b83bd2c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:59.371895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d091d7-cc56-4b92-a500-a61b83bd2c5e service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:59.381938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 762/1513] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:00.253727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecda7b-4501-42d8-a96a-87d60f125fdb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:00.254293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecda7b-4501-42d8-a96a-87d60f125fdb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:00.254293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecda7b-4501-42d8-a96a-87d60f125fdb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:00.255662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:00.260457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ec765-579a-4ed1-9a54-cec59f2af116 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:00.260922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ec765-579a-4ed1-9a54-cec59f2af116 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:00.261034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ec765-579a-4ed1-9a54-cec59f2af116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:00.280141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fc5ec765-579a-4ed1-9a54-cec59f2af116 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:00.281373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fc5ec765-579a-4ed1-9a54-cec59f2af116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:00.289290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:00.443777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecda7b-4501-42d8-a96a-87d60f125fdb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:00.458908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecda7b-4501-42d8-a96a-87d60f125fdb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:00.464452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c1985-f57a-44c1-ac17-c371165dc6ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:00.465341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c1985-f57a-44c1-ac17-c371165dc6ff service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:00.465625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c1985-f57a-44c1-ac17-c371165dc6ff service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:00.465902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c1985-f57a-44c1-ac17-c371165dc6ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:00.475114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c1985-f57a-44c1-ac17-c371165dc6ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:00.476712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c1985-f57a-44c1-ac17-c371165dc6ff service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:00.486590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 753/1516] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:54:00.893239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436acfa-e587-4940-9918-0ad275ef6f9d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:00.893451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436acfa-e587-4940-9918-0ad275ef6f9d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:00.893711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436acfa-e587-4940-9918-0ad275ef6f9d None None] PATH_INFO: `/v3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:00.895065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 764/1517] 10.222.0.82 () {54 vars in 925 bytes} [Thu May 26 13:54:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:54:00.901497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa98ab7-97ee-4fed-9898-c33321f3b712 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:00.901815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa98ab7-97ee-4fed-9898-c33321f3b712 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:00.902096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa98ab7-97ee-4fed-9898-c33321f3b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:00.922717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1fa98ab7-97ee-4fed-9898-c33321f3b712 None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:00.923928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1fa98ab7-97ee-4fed-9898-c33321f3b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:00.988308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 754/1518] 10.222.0.82 () {58 vars in 1015 bytes} [Thu May 26 13:54:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:10.274318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=37 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:10.395112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5377f7b-4ce5-49c7-9ce5-a8b3bc15d1bd None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:10.395311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5377f7b-4ce5-49c7-9ce5-a8b3bc15d1bd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:10.395578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5377f7b-4ce5-49c7-9ce5-a8b3bc15d1bd None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:10.396979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 765/1519] 10.222.0.91 () {54 vars in 960 bytes} [Thu May 26 13:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:10.403440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7240931-7ae5-4bfa-b831-a91e787c21bb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:10.403959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7240931-7ae5-4bfa-b831-a91e787c21bb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:10.403959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7240931-7ae5-4bfa-b831-a91e787c21bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:10.422589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d7240931-7ae5-4bfa-b831-a91e787c21bb None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:10.423778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7240931-7ae5-4bfa-b831-a91e787c21bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:10.432164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 755/1520] 10.222.0.91 () {58 vars in 1057 bytes} [Thu May 26 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:10.440167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5377f7b-4ce5-49c7-9ce5-a8b3bc15d1bd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:10.456951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5377f7b-4ce5-49c7-9ce5-a8b3bc15d1bd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:10.462266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b637e-5018-4a27-8166-1a8b4a5c692a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:10.462962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b637e-5018-4a27-8166-1a8b4a5c692a service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:10.463249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b637e-5018-4a27-8166-1a8b4a5c692a service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:10.463532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b637e-5018-4a27-8166-1a8b4a5c692a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:10.472200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b637e-5018-4a27-8166-1a8b4a5c692a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:10.473868 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b637e-5018-4a27-8166-1a8b4a5c692a service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:10.483810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 766/1521] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:11.762185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59a62-7652-43b9-a716-f51f3ee829d9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:11.763304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59a62-7652-43b9-a716-f51f3ee829d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:11.763449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59a62-7652-43b9-a716-f51f3ee829d9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:11.764786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:11.769578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30910b5-3971-4f1b-be07-a9e40b0a01cc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:11.770030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30910b5-3971-4f1b-be07-a9e40b0a01cc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:11.770151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30910b5-3971-4f1b-be07-a9e40b0a01cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:11.788096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a30910b5-3971-4f1b-be07-a9e40b0a01cc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:11.789301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a30910b5-3971-4f1b-be07-a9e40b0a01cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:11.797086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:11.951440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59a62-7652-43b9-a716-f51f3ee829d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:11.965686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59a62-7652-43b9-a716-f51f3ee829d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:11.971029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15f5f6-695e-4285-b988-bbb71de98e27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:11.971661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15f5f6-695e-4285-b988-bbb71de98e27 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:11.971943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15f5f6-695e-4285-b988-bbb71de98e27 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:11.972219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15f5f6-695e-4285-b988-bbb71de98e27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:11.980166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15f5f6-695e-4285-b988-bbb71de98e27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:11.982419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15f5f6-695e-4285-b988-bbb71de98e27 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:11.991637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 757/1524] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:12.859621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff919c-bc76-4cd9-b220-dafc916f27a5 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:12.860194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff919c-bc76-4cd9-b220-dafc916f27a5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:12.860194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff919c-bc76-4cd9-b220-dafc916f27a5 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:12.861639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:12.867236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9249b-e1dc-4b57-b200-cf3c23eacc31 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:12.867689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9249b-e1dc-4b57-b200-cf3c23eacc31 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:12.867785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9249b-e1dc-4b57-b200-cf3c23eacc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:12.885467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cec9249b-e1dc-4b57-b200-cf3c23eacc31 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:12.886647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cec9249b-e1dc-4b57-b200-cf3c23eacc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:12.895229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:13.045232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff919c-bc76-4cd9-b220-dafc916f27a5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:13.059462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff919c-bc76-4cd9-b220-dafc916f27a5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:13.064756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1962d-42d5-42a7-84f0-578b8130cfbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:13.065491 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1962d-42d5-42a7-84f0-578b8130cfbc service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:13.065999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1962d-42d5-42a7-84f0-578b8130cfbc service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:13.066510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1962d-42d5-42a7-84f0-578b8130cfbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:13.077264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1962d-42d5-42a7-84f0-578b8130cfbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:13.078895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1962d-42d5-42a7-84f0-578b8130cfbc service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:13.088451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 769/1527] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:17.221872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e518-8451-4f8d-9708-1789cca0add6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:17.222372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e518-8451-4f8d-9708-1789cca0add6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:17.222472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e518-8451-4f8d-9708-1789cca0add6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:17.223852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:17.229104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7620d-9f9f-467d-be44-7e596ea73212 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:17.229569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7620d-9f9f-467d-be44-7e596ea73212 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:17.229701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7620d-9f9f-467d-be44-7e596ea73212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:17.247838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-39a7620d-9f9f-467d-be44-7e596ea73212 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:17.249016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-39a7620d-9f9f-467d-be44-7e596ea73212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:17.256620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:17.293860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5e518-8451-4f8d-9708-1789cca0add6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:17.295611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bdb5e518-8451-4f8d-9708-1789cca0add6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:17.354246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5e518-8451-4f8d-9708-1789cca0add6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:17.357814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804047c1-a442-4b54-8264-a543c3be5169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:17.358458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804047c1-a442-4b54-8264-a543c3be5169 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:17.358742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804047c1-a442-4b54-8264-a543c3be5169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:17.359018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804047c1-a442-4b54-8264-a543c3be5169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:17.367876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804047c1-a442-4b54-8264-a543c3be5169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:17.369330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804047c1-a442-4b54-8264-a543c3be5169 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:17.374223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:17.379178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 760/1530] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:19.684087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cd8fd-9493-4b93-ae17-ed8a1cb73cf1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:19.685283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cd8fd-9493-4b93-ae17-ed8a1cb73cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:19.685283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cd8fd-9493-4b93-ae17-ed8a1cb73cf1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:19.686721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:19.691985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d3dc-c0b0-48c7-b509-c547857de643 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:19.692409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d3dc-c0b0-48c7-b509-c547857de643 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:19.692546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d3dc-c0b0-48c7-b509-c547857de643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:19.710217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-72b7d3dc-c0b0-48c7-b509-c547857de643 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:19.711424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-72b7d3dc-c0b0-48c7-b509-c547857de643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:19.719069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:19.755374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cd8fd-9493-4b93-ae17-ed8a1cb73cf1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:19.766014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cd8fd-9493-4b93-ae17-ed8a1cb73cf1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:19.769693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c551a-7484-4b80-b480-26c62452c0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:19.770328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c551a-7484-4b80-b480-26c62452c0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:19.770610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c551a-7484-4b80-b480-26c62452c0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:19.770888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c551a-7484-4b80-b480-26c62452c0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:19.779708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c551a-7484-4b80-b480-26c62452c0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:19.781139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c551a-7484-4b80-b480-26c62452c0d1 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:19.790582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 772/1533] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:23.562001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a01d-5ec6-404b-af95-a75ca3a8739f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:23.562537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a01d-5ec6-404b-af95-a75ca3a8739f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:23.562537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368a01d-5ec6-404b-af95-a75ca3a8739f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:23.563923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:23.569301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b282c-ec9b-42d8-9edc-d34b27849499 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:23.569715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b282c-ec9b-42d8-9edc-d34b27849499 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:23.569859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b282c-ec9b-42d8-9edc-d34b27849499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:23.588279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c26b282c-ec9b-42d8-9edc-d34b27849499 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:23.589512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c26b282c-ec9b-42d8-9edc-d34b27849499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:23.598028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:23.874057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368a01d-5ec6-404b-af95-a75ca3a8739f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:23.875732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1368a01d-5ec6-404b-af95-a75ca3a8739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:23.931383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368a01d-5ec6-404b-af95-a75ca3a8739f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:23.935498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d5f66-e6cf-4e17-be17-1fa9c8413dac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:23.936147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5f66-e6cf-4e17-be17-1fa9c8413dac service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:23.936432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5f66-e6cf-4e17-be17-1fa9c8413dac service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:23.936713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5f66-e6cf-4e17-be17-1fa9c8413dac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:23.945513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d5f66-e6cf-4e17-be17-1fa9c8413dac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:23.946935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d5f66-e6cf-4e17-be17-1fa9c8413dac service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:23.958389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 763/1536] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:26.187300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbde02-8f01-44ff-9135-6fc4f54a9a06 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:26.187888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbde02-8f01-44ff-9135-6fc4f54a9a06 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:26.187888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbde02-8f01-44ff-9135-6fc4f54a9a06 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:26.189251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:54:26.194484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a548942-ab23-4d2f-869d-a4db7ae69bc9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:26.194943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a548942-ab23-4d2f-869d-a4db7ae69bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:26.195544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a548942-ab23-4d2f-869d-a4db7ae69bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:26.212844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8a548942-ab23-4d2f-869d-a4db7ae69bc9 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:26.214212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a548942-ab23-4d2f-869d-a4db7ae69bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:26.222444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:26.500633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbde02-8f01-44ff-9135-6fc4f54a9a06 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:26.502484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6abbde02-8f01-44ff-9135-6fc4f54a9a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:26.515777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbde02-8f01-44ff-9135-6fc4f54a9a06 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:26.519284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedf9bd-8007-4951-a20b-e1f3d15e7f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:26.519922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf9bd-8007-4951-a20b-e1f3d15e7f0a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:26.520204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf9bd-8007-4951-a20b-e1f3d15e7f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:26.520480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf9bd-8007-4951-a20b-e1f3d15e7f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:26.530408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedf9bd-8007-4951-a20b-e1f3d15e7f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:26.532916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedf9bd-8007-4951-a20b-e1f3d15e7f0a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:26.543376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 775/1539] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:36.207638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=32 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:36.539055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=33 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:58.220979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf139a-9923-432c-bd60-f690ea2d5352 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.225454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf139a-9923-432c-bd60-f690ea2d5352 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.225454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf139a-9923-432c-bd60-f690ea2d5352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.256033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eddf139a-9923-432c-bd60-f690ea2d5352 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.257480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eddf139a-9923-432c-bd60-f690ea2d5352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.262321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2172b-3da6-44a1-a973-cbbcf713efc3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.263380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2172b-3da6-44a1-a973-cbbcf713efc3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.264059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2172b-3da6-44a1-a973-cbbcf713efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.268473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 765/1540] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.279141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450c18-5f08-4242-90c5-7701852863a4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.279453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450c18-5f08-4242-90c5-7701852863a4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.279704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450c18-5f08-4242-90c5-7701852863a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.292068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b5c2172b-3da6-44a1-a973-cbbcf713efc3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.293794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b5c2172b-3da6-44a1-a973-cbbcf713efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.306034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 776/1541] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.315559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f2450c18-5f08-4242-90c5-7701852863a4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.316602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922de39b-798b-4660-9dc0-8c8a04e4fe9c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.316803 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2450c18-5f08-4242-90c5-7701852863a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.317427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922de39b-798b-4660-9dc0-8c8a04e4fe9c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.318020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922de39b-798b-4660-9dc0-8c8a04e4fe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.342314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-922de39b-798b-4660-9dc0-8c8a04e4fe9c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.343454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 766/1542] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:55:58.347992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926100f2-184b-44c5-ba48-217540f39de4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.347992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926100f2-184b-44c5-ba48-217540f39de4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.347992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926100f2-184b-44c5-ba48-217540f39de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.348981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-922de39b-798b-4660-9dc0-8c8a04e4fe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.361510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 777/1543] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:55:58.362733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922de39b-798b-4660-9dc0-8c8a04e4fe9c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.384900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922de39b-798b-4660-9dc0-8c8a04e4fe9c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.386226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-926100f2-184b-44c5-ba48-217540f39de4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.393180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-926100f2-184b-44c5-ba48-217540f39de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.398146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.399187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.399715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.400213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.401182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:58.401811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.403172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:55:58.403172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) May 26 13:55:58.403172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: May 26 13:55:58.404955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.425790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 778/1544] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:58.426970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 767/1545] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.427298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.436528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926100f2-184b-44c5-ba48-217540f39de4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.443975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37401f8-e09d-4a0a-a590-1c5b615a4380 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.451467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f8f52-64b3-42e3-8743-39a7b4ef0694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.452614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8f52-64b3-42e3-8743-39a7b4ef0694 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.453093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926100f2-184b-44c5-ba48-217540f39de4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.453313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8f52-64b3-42e3-8743-39a7b4ef0694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.454168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f8f52-64b3-42e3-8743-39a7b4ef0694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.460353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f8f52-64b3-42e3-8743-39a7b4ef0694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:58.460353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f8f52-64b3-42e3-8743-39a7b4ef0694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.461361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f8f52-64b3-42e3-8743-39a7b4ef0694 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.468148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.469131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.470233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.470707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.471683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.472963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:55:58.472963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:55:58.472963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:55:58.474583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.479951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 779/1546] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:58.483128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982b979-c317-41e8-8a9b-10a043b43abd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.483316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982b979-c317-41e8-8a9b-10a043b43abd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.483509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982b979-c317-41e8-8a9b-10a043b43abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.516729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0982b979-c317-41e8-8a9b-10a043b43abd None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.518027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0982b979-c317-41e8-8a9b-10a043b43abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.537305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 780/1547] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.539094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced3f94-36f8-4941-b051-3ce6be230712 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.539589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced3f94-36f8-4941-b051-3ce6be230712 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.540053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced3f94-36f8-4941-b051-3ce6be230712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.544626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 768/1548] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/projects => generated 401 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:58.545799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.560107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3ced3f94-36f8-4941-b051-3ce6be230712 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.561687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3ced3f94-36f8-4941-b051-3ce6be230712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.646228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70670f80-5212-4bd6-b482-001732a920f4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.648705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 781/1549] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.655314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced3f94-36f8-4941-b051-3ce6be230712 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.656575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d54f55-d530-4e83-a0fc-8d79240f6d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.661689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d54f55-d530-4e83-a0fc-8d79240f6d4b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.662245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d54f55-d530-4e83-a0fc-8d79240f6d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.662734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d54f55-d530-4e83-a0fc-8d79240f6d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.663921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d54f55-d530-4e83-a0fc-8d79240f6d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.666112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d54f55-d530-4e83-a0fc-8d79240f6d4b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.667824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced3f94-36f8-4941-b051-3ce6be230712 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.673360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.674297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.674586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.674896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.675631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:58.675941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.677575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.691559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 782/1550] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:58.692345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.703641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 769/1551] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:58.706736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e47c6-a5f7-4267-aa17-a08eab79cc60 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.707353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e47c6-a5f7-4267-aa17-a08eab79cc60 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.707512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55b9b0-abae-488b-9654-85129d91b59b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.708052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e47c6-a5f7-4267-aa17-a08eab79cc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.725552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d08e47c6-a5f7-4267-aa17-a08eab79cc60 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.726766 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d08e47c6-a5f7-4267-aa17-a08eab79cc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.764148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311490e-b66d-40b4-b653-bafe9fe548eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.764831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311490e-b66d-40b4-b653-bafe9fe548eb admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.765118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311490e-b66d-40b4-b653-bafe9fe548eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.765430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311490e-b66d-40b4-b653-bafe9fe548eb admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.766215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311490e-b66d-40b4-b653-bafe9fe548eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.767762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-e311490e-b66d-40b4-b653-bafe9fe548eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:55:58.767762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) May 26 13:55:58.767762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: May 26 13:55:58.769176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311490e-b66d-40b4-b653-bafe9fe548eb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.787525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 770/1552] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.791815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 783/1553] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:58.791991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3ba4a-68f2-4336-9a7d-43db97dc4b59 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.791991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3ba4a-68f2-4336-9a7d-43db97dc4b59 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.791991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3ba4a-68f2-4336-9a7d-43db97dc4b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.792284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a446800-45ac-49da-9aa3-5769d4408a2e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.792284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a446800-45ac-49da-9aa3-5769d4408a2e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.792284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a446800-45ac-49da-9aa3-5769d4408a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.810798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c1a3ba4a-68f2-4336-9a7d-43db97dc4b59 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.811170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2a446800-45ac-49da-9aa3-5769d4408a2e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.811997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c1a3ba4a-68f2-4336-9a7d-43db97dc4b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.812388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a446800-45ac-49da-9aa3-5769d4408a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.820312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 771/1554] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.824675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 784/1555] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.824872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd2d4-53ab-477d-83f2-d4c07bc8a805 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.824987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a446800-45ac-49da-9aa3-5769d4408a2e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.825081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd2d4-53ab-477d-83f2-d4c07bc8a805 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.825081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd2d4-53ab-477d-83f2-d4c07bc8a805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.836550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a446800-45ac-49da-9aa3-5769d4408a2e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.841210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.841597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f23bd2d4-53ab-477d-83f2-d4c07bc8a805 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.841878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.842168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.842441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.843045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f23bd2d4-53ab-477d-83f2-d4c07bc8a805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.843156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.845049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.850941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 772/1556] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.852239 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ec526-2f17-4022-a160-2bd8088edebb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.852526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ec526-2f17-4022-a160-2bd8088edebb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.852968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ec526-2f17-4022-a160-2bd8088edebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.872211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 785/1557] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:58.873289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.874013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c13ec526-2f17-4022-a160-2bd8088edebb None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:58.875301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c13ec526-2f17-4022-a160-2bd8088edebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:58.878002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:58.903058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:58.941833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ced3-70fa-466e-ba52-dc677424d8b6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.946402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8fc1c-a44a-4728-8d12-781c9c6a23c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.947091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8fc1c-a44a-4728-8d12-781c9c6a23c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.947378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8fc1c-a44a-4728-8d12-781c9c6a23c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.947973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8fc1c-a44a-4728-8d12-781c9c6a23c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.948103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 773/1558] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:58.948485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8fc1c-a44a-4728-8d12-781c9c6a23c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.948607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ec526-2f17-4022-a160-2bd8088edebb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:58.950626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8fc1c-a44a-4728-8d12-781c9c6a23c9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.966078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ec526-2f17-4022-a160-2bd8088edebb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:58.972131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:58.973139 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.973674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.974073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.975175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:58.976004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 786/1559] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:58.976835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:58.977522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080c15-4adb-430a-b20f-a20c21e8bbca None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:58.977817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080c15-4adb-430a-b20f-a20c21e8bbca None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:58.978100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080c15-4adb-430a-b20f-a20c21e8bbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:58.995011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 774/1560] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:58.995011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.002473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-41080c15-4adb-430a-b20f-a20c21e8bbca None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:59.003853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-41080c15-4adb-430a-b20f-a20c21e8bbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:59.005067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d136a4b-23d0-44d7-b2f5-78100acc3e2c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.011767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed715e-fee6-4c26-adca-30c7fbcea9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.012718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 787/1561] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:59.012856 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed715e-fee6-4c26-adca-30c7fbcea9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.013186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed715e-fee6-4c26-adca-30c7fbcea9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.013625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed715e-fee6-4c26-adca-30c7fbcea9ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.014004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda2c7d-93c7-4772-b78c-cb5f2b7fe0d6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.014291 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda2c7d-93c7-4772-b78c-cb5f2b7fe0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.014569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda2c7d-93c7-4772-b78c-cb5f2b7fe0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.014767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed715e-fee6-4c26-adca-30c7fbcea9ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:59.015249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed715e-fee6-4c26-adca-30c7fbcea9ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.017897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed715e-fee6-4c26-adca-30c7fbcea9ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.033326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5eda2c7d-93c7-4772-b78c-cb5f2b7fe0d6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:59.034613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5eda2c7d-93c7-4772-b78c-cb5f2b7fe0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:59.036319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 775/1562] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:59.038189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e08870-cb1d-42f2-8055-5af40d8ea834 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.038629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e08870-cb1d-42f2-8055-5af40d8ea834 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.039082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e08870-cb1d-42f2-8055-5af40d8ea834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.048122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 788/1563] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:59.048756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda2c7d-93c7-4772-b78c-cb5f2b7fe0d6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.058197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda2c7d-93c7-4772-b78c-cb5f2b7fe0d6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.061207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c5e08870-cb1d-42f2-8055-5af40d8ea834 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:59.062466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c5e08870-cb1d-42f2-8055-5af40d8ea834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:59.063346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.063574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.063862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.064143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.064837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.066970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.071413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 776/1564] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:59.072492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dede024-7290-4fdf-a72b-5a9bd8deb13e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.072781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dede024-7290-4fdf-a72b-5a9bd8deb13e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.073059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dede024-7290-4fdf-a72b-5a9bd8deb13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.089207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 789/1565] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.090003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.093314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6dede024-7290-4fdf-a72b-5a9bd8deb13e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:59.095159 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6dede024-7290-4fdf-a72b-5a9bd8deb13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:59.101952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675444c1-69ac-4daf-a4b8-d288a3cab139 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.103397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 777/1566] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:59.104300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dede024-7290-4fdf-a72b-5a9bd8deb13e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.106936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.107658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.107943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.108220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.108941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.110593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.113697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dede024-7290-4fdf-a72b-5a9bd8deb13e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.118827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.119493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.119777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.120054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.120647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 790/1567] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:59.120914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.122627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.124172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.134379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d652581-12d3-44f6-a602-f6769636de7c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.134823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 778/1568] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:59.135703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.143339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.144010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.144823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.144823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.145328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:59.145643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.147323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.148102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b467e-dec2-469c-bc06-2e030b3f7a07 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.152689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.153387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.153672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.154201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.154890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.157546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.160855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 791/1569] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:59.161867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.171767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcdd75-0510-4c9c-be71-bb7f93509c32 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.176636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.177333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.177619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.177913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.178607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:59.178917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.180551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.190535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 779/1570] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.191373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.196718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 792/1571] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:59.196815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.207014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb4bef-17f0-4e98-9721-f5da8c281ff2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.259074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e49b0-69c0-4f35-b86f-2ade84b80c4a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.264275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.264949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.265437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.265437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.265677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.265839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.266525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.266890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.267006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:59.267288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.267619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.268888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.269277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.279514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 793/1572] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:59.280329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.283973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 780/1573] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:59.283973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.292748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c85fab-6d5a-4f7b-9f2c-cf204d622433 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.292969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc194b-aa95-4de3-92fc-6facb4144001 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.297579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.297738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.298232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.298368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.298516 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.298641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.298791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.298901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.299511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.299629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:59.299813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.301102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.301441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.311821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 781/1574] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:59.312857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.322302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 794/1575] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:55:59.322302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.323742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac5765-3b87-4b23-a375-b92cb73df66c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.329597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.330483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.330767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.331067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] PATH_INFO: `/v3/projects/73c3e29dafa248a3bce455dfd308ab7c/users/558985b6382c4647b5f734d112f9b5fa/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.333407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d583b66-873a-4c15-972e-a1288dda9eab admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.339947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.340677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.340964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.341280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.341987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.345270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.347026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c3e29dafa248a3bce455dfd308ab7c, user_id=558985b6382c4647b5f734d112f9b5fa, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.348072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:55:59.348072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:55:59.348072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:55:59.349199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:59.350164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.364973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 782/1576] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:55:59 2022] PUT /identity/v3/projects/73c3e29dafa248a3bce455dfd308ab7c/users/558985b6382c4647b5f734d112f9b5fa/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:55:59.366840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.377533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 795/1577] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.378688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.393177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:59.435869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709df17-7919-47ac-9739-8ed45c976172 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.441004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.441705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.441814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd37a-d028-4d47-b475-b7e06f3911c7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.441990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.442265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.442958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.444872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.446723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.447392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.447713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.448215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.448946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.450933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.466491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 783/1578] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.470603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.480166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd62067-506e-49c3-9f0c-a058342411a1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.483505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 796/1579] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.483505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.539094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.539691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.539979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.540254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.541003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.542655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.544039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0a874-a604-4363-90eb-8599baa0e12f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.548686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.549508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.549802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.550085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.550800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.552520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 784/1580] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:59.552732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.553333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.562588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4f38-4c98-4777-83c9-9fc1b8d2d92c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.568217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae85bc-bdb6-4574-9183-a007bee463b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.569270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae85bc-bdb6-4574-9183-a007bee463b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.569714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae85bc-bdb6-4574-9183-a007bee463b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.569942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae85bc-bdb6-4574-9183-a007bee463b1 admin admin] PATH_INFO: `/v3/projects/67bf852b45d8419aad80b70b02e2bf29/users/b6e1ee20eef14ec3bf13d7473f28fbcb/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.579881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 797/1581] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.580708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.586324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae85bc-bdb6-4574-9183-a007bee463b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bf852b45d8419aad80b70b02e2bf29, user_id=b6e1ee20eef14ec3bf13d7473f28fbcb, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.589112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae85bc-bdb6-4574-9183-a007bee463b1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.603155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 785/1582] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:55:59 2022] PUT /identity/v3/projects/67bf852b45d8419aad80b70b02e2bf29/users/b6e1ee20eef14ec3bf13d7473f28fbcb/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:55:59.604435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9cab6-f017-482d-b091-ae4c3e3a50d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.604730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9cab6-f017-482d-b091-ae4c3e3a50d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.605011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9cab6-f017-482d-b091-ae4c3e3a50d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.633550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fbd9cab6-f017-482d-b091-ae4c3e3a50d5 None None] MFA Rules not processed for user `558985b6382c4647b5f734d112f9b5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:59.634783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbd9cab6-f017-482d-b091-ae4c3e3a50d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:59.640664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:59.646496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220f1ee-059d-4de9-8b8c-7742ce4c6a64 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.703432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 786/1583] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:59.704959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9cab6-f017-482d-b091-ae4c3e3a50d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.707733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.708422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.708692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.708975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.709710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.711623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.715238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9cab6-f017-482d-b091-ae4c3e3a50d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.719987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.720649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.720935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.721274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.721963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.723915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.733081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:59.737806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 798/1584] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.738850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.746541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 787/1585] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.747236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.757771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52618755-948a-4721-9e8c-be20ffeb920a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.803281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491eba3-59f3-42dc-96be-a8e41345ebe5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.808207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.808866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.809170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.809453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.810186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.811783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.814152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.814831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.815115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.815389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.816079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.818016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.821992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 799/1586] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:59.822844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.833308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6a601-6120-46f8-aaf8-b7554c9a360c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.838559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db941d7-72ad-4f0f-8e29-21bcd4a8881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.839637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db941d7-72ad-4f0f-8e29-21bcd4a8881d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.839786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 788/1587] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.839930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db941d7-72ad-4f0f-8e29-21bcd4a8881d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.840192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db941d7-72ad-4f0f-8e29-21bcd4a8881d admin admin] PATH_INFO: `/v3/projects/7116b23d173c4192a6c8d996ad31d57c/users/c67205b7150c460a81ff545d5ba29cad/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.840545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.851784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e508a7-8728-446b-8ef4-55619b78119a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.856591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.857275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.857468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db941d7-72ad-4f0f-8e29-21bcd4a8881d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7116b23d173c4192a6c8d996ad31d57c, user_id=c67205b7150c460a81ff545d5ba29cad, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.857581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.857829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.858510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.860781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db941d7-72ad-4f0f-8e29-21bcd4a8881d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.860928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.874613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 800/1588] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:55:59 2022] PUT /identity/v3/projects/7116b23d173c4192a6c8d996ad31d57c/users/c67205b7150c460a81ff545d5ba29cad/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:55:59.876598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e52b0-6805-4179-9c3e-ada91e5b65c5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.876884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e52b0-6805-4179-9c3e-ada91e5b65c5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.877170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e52b0-6805-4179-9c3e-ada91e5b65c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.888007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 789/1589] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.888007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.895658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46188a10-e748-4311-8f4f-0ad71e76f447 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.903268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-046e52b0-6805-4179-9c3e-ada91e5b65c5 None None] MFA Rules not processed for user `b6e1ee20eef14ec3bf13d7473f28fbcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:59.904781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-046e52b0-6805-4179-9c3e-ada91e5b65c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:59.951437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:59.953029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.954065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.954282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.954657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.955269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.957259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:59.971123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 801/1590] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:59.972141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e52b0-6805-4179-9c3e-ada91e5b65c5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.983588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e52b0-6805-4179-9c3e-ada91e5b65c5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:59.988751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:59.989458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:59.989744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:59.990277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:59.990850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:59.991309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 790/1591] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/projects => generated 401 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:59.992179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:59.992792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.019088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 802/1592] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:55:59 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:00.019926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.030442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c7e83-703d-4321-86da-6f87687a6385 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.050752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:00.063557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de94f7-3980-4a2b-8868-c0f7362e81ea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.068306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.068975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.069301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.069580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.070304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.071910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.082479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 791/1593] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.083592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.090387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3052eb-c97f-4544-b98a-f5a5f5a99a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.091067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3052eb-c97f-4544-b98a-f5a5f5a99a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.091351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3052eb-c97f-4544-b98a-f5a5f5a99a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.091636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3052eb-c97f-4544-b98a-f5a5f5a99a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.092410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3052eb-c97f-4544-b98a-f5a5f5a99a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.094045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3052eb-c97f-4544-b98a-f5a5f5a99a78 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.094201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c41be8-9d44-4b7d-9056-e1fbc30d2bc6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.098583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.099225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.099507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.099804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.100508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.102083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.104356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 803/1594] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.105518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3e31a-5e21-47df-9945-19e0bd967102 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.105816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3e31a-5e21-47df-9945-19e0bd967102 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.106094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3e31a-5e21-47df-9945-19e0bd967102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.116381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 792/1595] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.116381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.124340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-39e3e31a-5e21-47df-9945-19e0bd967102 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:00.125266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68b01-ea68-40f9-b2b3-6a2e4ecee1d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.125643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-39e3e31a-5e21-47df-9945-19e0bd967102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:00.130522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab03ca3-f5f0-4cf6-93ae-276353793ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.131177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab03ca3-f5f0-4cf6-93ae-276353793ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.131460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab03ca3-f5f0-4cf6-93ae-276353793ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.131734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab03ca3-f5f0-4cf6-93ae-276353793ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.132482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab03ca3-f5f0-4cf6-93ae-276353793ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.134037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 804/1596] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:00.134263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab03ca3-f5f0-4cf6-93ae-276353793ed5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.134810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3e31a-5e21-47df-9945-19e0bd967102 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.144830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3e31a-5e21-47df-9945-19e0bd967102 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.145001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 793/1597] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.146213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b5baa-7699-4499-b7b9-f81440d45fa2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.146499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b5baa-7699-4499-b7b9-f81440d45fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.146771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b5baa-7699-4499-b7b9-f81440d45fa2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.148268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 794/1598] 10.222.0.91 () {54 vars in 961 bytes} [Thu May 26 13:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:00.148958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b5baa-7699-4499-b7b9-f81440d45fa2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.151513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.152186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.152476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.152754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.153485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.155670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.162379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b5baa-7699-4499-b7b9-f81440d45fa2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.167072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.167899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.168275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.168656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.169617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.171366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.178206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 805/1599] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:00 2022] POST /identity/v3/users => generated 307 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:00.178206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.184855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 795/1600] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.185703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.189610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee961c-fbf6-4a03-ba41-27ba5096d56b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.194885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.195492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.195771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.196041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.196764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.196869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3773-21c3-46e6-8db5-44ed572f830d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.198382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.201522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.202512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.202807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.203087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] PATH_INFO: `/v3/projects/e3d27b9eb0ef4c338c18c2a1d02db444/users/8d7bcc48c8d844e5b36b1c4e2748e766/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.209133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 806/1601] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.209961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.218689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d27b9eb0ef4c338c18c2a1d02db444, user_id=8d7bcc48c8d844e5b36b1c4e2748e766, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.219981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b1edc-b0bf-4729-b221-9e6535bc6cb7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.221948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.224776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.225495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.225923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.226168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.226922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.228567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.235953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 796/1602] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:00 2022] PUT /identity/v3/projects/e3d27b9eb0ef4c338c18c2a1d02db444/users/8d7bcc48c8d844e5b36b1c4e2748e766/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:00.236765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.246038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 807/1603] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.246038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.253333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b1eb-cd9e-478c-b891-04adb4a51a09 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.290668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1dfc7-826e-47e8-9949-313f2e32afb7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.294286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.294936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.295223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.295500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.310938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4891d9b-ba8a-44d6-a4db-3930e5f07a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.311606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891d9b-ba8a-44d6-a4db-3930e5f07a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.311887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891d9b-ba8a-44d6-a4db-3930e5f07a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.312160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891d9b-ba8a-44d6-a4db-3930e5f07a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.312939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4891d9b-ba8a-44d6-a4db-3930e5f07a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.314793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4891d9b-ba8a-44d6-a4db-3930e5f07a00 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.324658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 808/1604] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.326071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229b416-b966-411b-8c39-948d8eb4fe7e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.326366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229b416-b966-411b-8c39-948d8eb4fe7e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.326645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229b416-b966-411b-8c39-948d8eb4fe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.345746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e229b416-b966-411b-8c39-948d8eb4fe7e None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:00.346945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e229b416-b966-411b-8c39-948d8eb4fe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:00.355993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.357598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.358873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:00.370069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 797/1605] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:00.370903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.383673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e2ed0-e586-42ed-b4cd-ee4aa43ea791 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.389097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.394407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.395050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.395340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.396365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.398915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.413594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 798/1606] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.417171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.426916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 809/1607] 10.222.0.91 () {58 vars in 1058 bytes} [Thu May 26 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:00.427764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229b416-b966-411b-8c39-948d8eb4fe7e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.429222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b8938-2d13-4079-a41a-3d94ffb09ef7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.433684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db05cd-8840-44d0-b804-981e21ec31e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.434347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db05cd-8840-44d0-b804-981e21ec31e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.434627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db05cd-8840-44d0-b804-981e21ec31e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.434906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db05cd-8840-44d0-b804-981e21ec31e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.435656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db05cd-8840-44d0-b804-981e21ec31e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.437316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db05cd-8840-44d0-b804-981e21ec31e8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.440105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229b416-b966-411b-8c39-948d8eb4fe7e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.445522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.446639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.447271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.447738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] PATH_INFO: `/v3/projects/74e1eca47d504eb887f1c9813f537010/users/34e66e352264450b9f99e2f710e6ba91/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.458047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 799/1608] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:00.458047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3e0f1-ed8d-4037-9ac2-e96ae7578324 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.458047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3e0f1-ed8d-4037-9ac2-e96ae7578324 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.458047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3e0f1-ed8d-4037-9ac2-e96ae7578324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.469779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e1eca47d504eb887f1c9813f537010, user_id=34e66e352264450b9f99e2f710e6ba91, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.472780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.476468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a9c3e0f1-ed8d-4037-9ac2-e96ae7578324 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:00.477709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9c3e0f1-ed8d-4037-9ac2-e96ae7578324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:00.486529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 800/1609] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:00.487339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3e0f1-ed8d-4037-9ac2-e96ae7578324 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.493134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 810/1610] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:00 2022] PUT /identity/v3/projects/74e1eca47d504eb887f1c9813f537010/users/34e66e352264450b9f99e2f710e6ba91/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:00.493823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.509188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3e0f1-ed8d-4037-9ac2-e96ae7578324 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.518778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:00.578150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.579041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479658-a287-4364-974a-b196326ae728 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.579167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.579339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.579580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] PATH_INFO: `/v3/projects/dadb5f6b57c04d5d8cdf41a03824eb3b/users/4ab0abfc9bb24cbebc5c04adcec47c10/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.586386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.587692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.588064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.588730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] PATH_INFO: `/v3/projects/af9d168c40ea4df99b495eb06245d066/users/592975b43454407abde6c7cab2504982/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.597023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dadb5f6b57c04d5d8cdf41a03824eb3b, user_id=4ab0abfc9bb24cbebc5c04adcec47c10, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.599849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.612737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9d168c40ea4df99b495eb06245d066, user_id=592975b43454407abde6c7cab2504982, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.615466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.616674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 801/1611] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:00 2022] PUT /identity/v3/projects/dadb5f6b57c04d5d8cdf41a03824eb3b/users/4ab0abfc9bb24cbebc5c04adcec47c10/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:00.627858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.637902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 811/1612] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:00 2022] PUT /identity/v3/projects/af9d168c40ea4df99b495eb06245d066/users/592975b43454407abde6c7cab2504982/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:00.638535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.707278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0085822-f0a3-461f-b153-74d3a2afe221 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.715589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80d057-b9bb-4037-8fd9-c29290ce66dd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.722270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.722939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.723273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.723700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.729946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:00.791033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.792398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.801958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.802867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.803149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.803456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] PATH_INFO: `/v3/projects/27a095273294426eb873c4dbf544e23f/users/d53596339a8a468ea69838eb7605e670/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.806174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 812/1613] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:00.807151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.820895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798854-6acd-4d11-9a70-584c8a431950 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:00.826904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd70711-b48d-410f-8293-11bd03355356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:00.828592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd70711-b48d-410f-8293-11bd03355356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.828883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd70711-b48d-410f-8293-11bd03355356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.829180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd70711-b48d-410f-8293-11bd03355356 admin admin] PATH_INFO: `/v3/projects/9ed49180e7f24cb296141d06576ead81/users/bda029aace2c4e40bcc470138243d0da/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.841304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a095273294426eb873c4dbf544e23f, user_id=d53596339a8a468ea69838eb7605e670, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.844226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.859131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd70711-b48d-410f-8293-11bd03355356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed49180e7f24cb296141d06576ead81, user_id=bda029aace2c4e40bcc470138243d0da, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:00.864048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd70711-b48d-410f-8293-11bd03355356 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:00.872760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 802/1614] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:00 2022] PUT /identity/v3/projects/27a095273294426eb873c4dbf544e23f/users/d53596339a8a468ea69838eb7605e670/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:00.872760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:00.874298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:00.886178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 813/1615] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:00 2022] PUT /identity/v3/projects/9ed49180e7f24cb296141d06576ead81/users/bda029aace2c4e40bcc470138243d0da/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:00.888628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1438fb-859c-4dd8-ac83-f27c8018204a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:00.889497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1438fb-859c-4dd8-ac83-f27c8018204a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:00.890019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1438fb-859c-4dd8-ac83-f27c8018204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:00.919359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:00.925415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2a1438fb-859c-4dd8-ac83-f27c8018204a None None] MFA Rules not processed for user `34e66e352264450b9f99e2f710e6ba91`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:00.927309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a1438fb-859c-4dd8-ac83-f27c8018204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:00.998897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13f58d-8bc0-4a5e-87ba-93f27c57c238 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.052194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 814/1616] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:01.054827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a303e4-0e0c-456f-9e3f-3bcc56bf4ec8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.055292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a303e4-0e0c-456f-9e3f-3bcc56bf4ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.056494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a303e4-0e0c-456f-9e3f-3bcc56bf4ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.093245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:01.093884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.094902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.095016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.112870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b3a303e4-0e0c-456f-9e3f-3bcc56bf4ec8 None None] MFA Rules not processed for user `4ab0abfc9bb24cbebc5c04adcec47c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:01.114306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b3a303e4-0e0c-456f-9e3f-3bcc56bf4ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:01.212917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:01.214424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:01.220276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:01.234313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 815/1617] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:01.246588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386414-ef2a-4fbc-b91f-7d5570075cc3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.246882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386414-ef2a-4fbc-b91f-7d5570075cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.247166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386414-ef2a-4fbc-b91f-7d5570075cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.252927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 803/1618] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:01.258050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:01.283942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79be30-5de8-4cf5-b9de-b3f217319482 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.291428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bf386414-ef2a-4fbc-b91f-7d5570075cc3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:01.294938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bf386414-ef2a-4fbc-b91f-7d5570075cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:01.298278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:01.300692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.301108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.302472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.303504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:01.307636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:01.320107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 816/1619] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:01.320928 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb282286-1285-49ee-9972-3f934cd249a7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.322385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb282286-1285-49ee-9972-3f934cd249a7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.322693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb282286-1285-49ee-9972-3f934cd249a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.333129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 804/1620] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:01.341631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:01.367364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cb282286-1285-49ee-9972-3f934cd249a7 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:01.368724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cb282286-1285-49ee-9972-3f934cd249a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:01.381246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d8c1f-19b4-4bac-b3e5-8d186b9e615e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.384441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 817/1621] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:01.388305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb282286-1285-49ee-9972-3f934cd249a7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:01.396402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:01.398042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.398842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.399800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.403591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:01.408831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:01.470197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 805/1622] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:01 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:01.471164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:01.513437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb282286-1285-49ee-9972-3f934cd249a7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.624067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:01.624831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.625121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.625530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.626792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317623d-ebbe-45f2-9e35-de57cc05b4a2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.632664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:01.634197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.634649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.634932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.652609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:01.744415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:01.746642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:01.753510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:01.755605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:01.769813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 806/1623] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:01.770962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:01.773434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 818/1624] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:01.774470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:01.795165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606df6-9119-4a8c-b50b-e75df1a360c9 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.803672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:01.804346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.804630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.804907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.807033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0443-24e9-4443-bed3-4be2474f8157 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.820294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9774b9-3973-4999-bb84-8c10ce5ce319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:01.823691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9774b9-3973-4999-bb84-8c10ce5ce319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.824010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9774b9-3973-4999-bb84-8c10ce5ce319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.825452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9774b9-3973-4999-bb84-8c10ce5ce319 admin admin] PATH_INFO: `/v3/projects/27a095273294426eb873c4dbf544e23f/users/d53596339a8a468ea69838eb7605e670/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.828229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9774b9-3973-4999-bb84-8c10ce5ce319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a095273294426eb873c4dbf544e23f, user_id=d53596339a8a468ea69838eb7605e670, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:01.830903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:01.832385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:01.838810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9774b9-3973-4999-bb84-8c10ce5ce319 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:01.857756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 819/1626] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:01.858544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:01.865611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 807/1626] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:01 2022] PUT /identity/v3/projects/27a095273294426eb873c4dbf544e23f/users/d53596339a8a468ea69838eb7605e670/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:01.867396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e884c-a643-409a-84f6-5ecb1777f4b8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.867694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e884c-a643-409a-84f6-5ecb1777f4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.867964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e884c-a643-409a-84f6-5ecb1777f4b8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.870578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 808/1627] 10.222.0.91 () {54 vars in 961 bytes} [Thu May 26 13:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:01.871915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a0a5d-9b7e-45b4-90f6-51395c91f06f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:01.872209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a0a5d-9b7e-45b4-90f6-51395c91f06f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:01.872551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a0a5d-9b7e-45b4-90f6-51395c91f06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:01.893191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91f48b-045a-42fa-a63e-bb9c09ea53fd service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:01.946809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d97a0a5d-9b7e-45b4-90f6-51395c91f06f None None] MFA Rules not processed for user `d53596339a8a468ea69838eb7605e670`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:01.948906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d97a0a5d-9b7e-45b4-90f6-51395c91f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:02.008174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:02.053131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15583a87-1562-4e07-88d9-9649a4eaa728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:02.057174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583a87-1562-4e07-88d9-9649a4eaa728 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.058331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583a87-1562-4e07-88d9-9649a4eaa728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.059676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583a87-1562-4e07-88d9-9649a4eaa728 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.061788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15583a87-1562-4e07-88d9-9649a4eaa728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:02.067391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15583a87-1562-4e07-88d9-9649a4eaa728 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:02.124745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 820/1628] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:01 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:02.128457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1eed-2e2d-49d4-9dbc-fd2fe1e7d59f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.129940 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1eed-2e2d-49d4-9dbc-fd2fe1e7d59f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.130235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1eed-2e2d-49d4-9dbc-fd2fe1e7d59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.155514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 809/1629] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:02.158628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a0a5d-9b7e-45b4-90f6-51395c91f06f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:02.199300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9d3d1eed-2e2d-49d4-9dbc-fd2fe1e7d59f None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:02.201941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a0a5d-9b7e-45b4-90f6-51395c91f06f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:02.202493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9d3d1eed-2e2d-49d4-9dbc-fd2fe1e7d59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:02.216226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:02.217981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.225418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.225865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.226546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:02.230140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:02.276578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 810/1630] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:02.283036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:02.415403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 821/1631] 10.222.0.91 () {58 vars in 1058 bytes} [Thu May 26 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:02.416469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1eed-2e2d-49d4-9dbc-fd2fe1e7d59f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:02.423083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e109e-c482-43b2-b50f-8e521ae3385e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:02.433497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0358245-a778-4c03-9237-f824e4497216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:02.434257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358245-a778-4c03-9237-f824e4497216 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.435291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358245-a778-4c03-9237-f824e4497216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.435662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358245-a778-4c03-9237-f824e4497216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.446344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1eed-2e2d-49d4-9dbc-fd2fe1e7d59f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:02.458371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:02.459900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.460383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.460727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] PATH_INFO: `/v3/projects/f223017dc9f7489a8089463f44fd53a9/users/23a4247590974d0a8ef02afd828edd26/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.463857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0358245-a778-4c03-9237-f824e4497216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:02.466755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0358245-a778-4c03-9237-f824e4497216 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:02.481854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f223017dc9f7489a8089463f44fd53a9, user_id=23a4247590974d0a8ef02afd828edd26, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:02.491682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:02.493741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 811/1632] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:02.495339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977489e-aa72-4bf5-a1b4-5239dba7fc69 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.495767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977489e-aa72-4bf5-a1b4-5239dba7fc69 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.496484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977489e-aa72-4bf5-a1b4-5239dba7fc69 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.499171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 812/1633] 10.222.0.91 () {54 vars in 912 bytes} [Thu May 26 13:56:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:02.506657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1897-c211-44c5-98ec-94e943ecfb19 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.506953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1897-c211-44c5-98ec-94e943ecfb19 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.507236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1897-c211-44c5-98ec-94e943ecfb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.560167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 822/1634] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:02 2022] PUT /identity/v3/projects/f223017dc9f7489a8089463f44fd53a9/users/23a4247590974d0a8ef02afd828edd26/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:02.563472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-70bc1897-c211-44c5-98ec-94e943ecfb19 None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:02.563708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:02.567151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-70bc1897-c211-44c5-98ec-94e943ecfb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:02.580376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:02.707297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dfc6b-a28f-4543-92bd-75914a97816f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:02.719926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:02.720628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.722187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.722629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.725144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:02.728352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:02.752281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 823/1635] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:02.754545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:02.795368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 813/1636] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:02.796896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc1897-c211-44c5-98ec-94e943ecfb19 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:02.822523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc1897-c211-44c5-98ec-94e943ecfb19 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:02.836966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72436821-390c-462d-9b48-2a3557ca9ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:02.838590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72436821-390c-462d-9b48-2a3557ca9ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.839203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72436821-390c-462d-9b48-2a3557ca9ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.839203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72436821-390c-462d-9b48-2a3557ca9ff7 admin admin] PATH_INFO: `/v3/projects/f223017dc9f7489a8089463f44fd53a9/users/23a4247590974d0a8ef02afd828edd26/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.847828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72436821-390c-462d-9b48-2a3557ca9ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f223017dc9f7489a8089463f44fd53a9, user_id=23a4247590974d0a8ef02afd828edd26, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:02.848604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:02.855127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72436821-390c-462d-9b48-2a3557ca9ff7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:02.892678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 814/1637] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:02 2022] PUT /identity/v3/projects/f223017dc9f7489a8089463f44fd53a9/users/23a4247590974d0a8ef02afd828edd26/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:02.896287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f903-7a35-484a-9791-642cc091d963 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:02.896555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af90de-0748-4a02-b8c4-cf168acd52b5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:02.896690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f903-7a35-484a-9791-642cc091d963 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:02.896960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f903-7a35-484a-9791-642cc091d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:02.960554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-58d5f903-7a35-484a-9791-642cc091d963 None None] MFA Rules not processed for user `23a4247590974d0a8ef02afd828edd26`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:02.961950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-58d5f903-7a35-484a-9791-642cc091d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:03.030627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:03.030884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:03.031157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:03.031460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:03.109536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:03.111864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:03.125209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 824/1638] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:03.125671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 815/1639] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:03.126153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:03.136776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe6471-f522-4fc6-bbb0-cda4ce854e2d service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:03.141776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:03.142444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:03.142788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:03.143062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:03.243500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:03.245170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:03.271288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 825/1640] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:03.400535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5f903-7a35-484a-9791-642cc091d963 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:03.414790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5f903-7a35-484a-9791-642cc091d963 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:03.419459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659cf4a-9dc6-4528-94b6-527f27481541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:03.420286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659cf4a-9dc6-4528-94b6-527f27481541 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:03.420649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659cf4a-9dc6-4528-94b6-527f27481541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:03.421060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659cf4a-9dc6-4528-94b6-527f27481541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:03.436554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659cf4a-9dc6-4528-94b6-527f27481541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:03.438137 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659cf4a-9dc6-4528-94b6-527f27481541 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:03.463406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 816/1641] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:03.639036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:03.648989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853e0e8-3b85-4e96-8993-64112ffa6cca service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:03.654663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7171e5e-48ad-411a-af0d-de563ce1424a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:03.654996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7171e5e-48ad-411a-af0d-de563ce1424a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:03.655276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7171e5e-48ad-411a-af0d-de563ce1424a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:03.655552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7171e5e-48ad-411a-af0d-de563ce1424a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:03.666828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3e2da-fe2d-4e71-9690-50634b3a3fc9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:03.667485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3e2da-fe2d-4e71-9690-50634b3a3fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:03.668465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3e2da-fe2d-4e71-9690-50634b3a3fc9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:03.671127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 817/1642] 10.222.0.91 () {54 vars in 912 bytes} [Thu May 26 13:56:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:56:03.677953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c145-0a48-4157-909c-96ed55258041 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:03.678528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c145-0a48-4157-909c-96ed55258041 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:03.679648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c145-0a48-4157-909c-96ed55258041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:03.708160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:03.713804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1005c145-0a48-4157-909c-96ed55258041 None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:03.716079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1005c145-0a48-4157-909c-96ed55258041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:03.732248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 818/1643] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 26 13:56:03.776655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7171e5e-48ad-411a-af0d-de563ce1424a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:03.779135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7171e5e-48ad-411a-af0d-de563ce1424a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:03.793232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 826/1644] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:04.002951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c145-0a48-4157-909c-96ed55258041 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:04.021629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c145-0a48-4157-909c-96ed55258041 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:04.034603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:04.035334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:04.035623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:04.035898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:04.061371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:04.069604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:04.100498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 819/1645] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:04.121852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6b62e-879d-4859-9615-accd03cf361b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:04.122685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6b62e-879d-4859-9615-accd03cf361b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:04.123240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6b62e-879d-4859-9615-accd03cf361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:04.172588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:04.223406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-19a6b62e-879d-4859-9615-accd03cf361b None None] MFA Rules not processed for user `bda029aace2c4e40bcc470138243d0da`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:04.227153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-19a6b62e-879d-4859-9615-accd03cf361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:04.293028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:04.330528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001533e8-a98e-477d-8f47-0d650d7d6de2 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:04.345865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:04.347638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:04.349285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:04.349731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] PATH_INFO: `/v3/users/b6e1ee20eef14ec3bf13d7473f28fbcb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:04.352621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e1ee20eef14ec3bf13d7473f28fbcb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:04.357339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:56:04.357339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 26 13:56:04.357339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: May 26 13:56:04.362100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:04.412168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 827/1646] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:04.424738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d984-52b5-4f42-be4d-be70182ce1cb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:04.425203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d984-52b5-4f42-be4d-be70182ce1cb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:04.425694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d984-52b5-4f42-be4d-be70182ce1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:04.462869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c915d984-52b5-4f42-be4d-be70182ce1cb None None] MFA Rules not processed for user `bda029aace2c4e40bcc470138243d0da`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:04.464361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c915d984-52b5-4f42-be4d-be70182ce1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:04.490090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 828/1647] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:04.503653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d984-52b5-4f42-be4d-be70182ce1cb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:04.530236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d984-52b5-4f42-be4d-be70182ce1cb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:04.536614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e1ee20eef14ec3bf13d7473f28fbcb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:04.548896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42ad40-b844-4b5d-8e7b-db94302f3970 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:04.549120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ad40-b844-4b5d-8e7b-db94302f3970 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:04.549468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ad40-b844-4b5d-8e7b-db94302f3970 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:04.549747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ad40-b844-4b5d-8e7b-db94302f3970 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:04.569466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42ad40-b844-4b5d-8e7b-db94302f3970 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:04.571276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42ad40-b844-4b5d-8e7b-db94302f3970 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:04.582296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e1ee20eef14ec3bf13d7473f28fbcb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:04.593738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 829/1648] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:04.626699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e1ee20eef14ec3bf13d7473f28fbcb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:04.627759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e1ee20eef14ec3bf13d7473f28fbcb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:04.628719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e1ee20eef14ec3bf13d7473f28fbcb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:04.649620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 820/1649] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:04 2022] DELETE /identity/v3/users/b6e1ee20eef14ec3bf13d7473f28fbcb => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:04.658028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea434e-b8cf-4022-9580-c31948d3f91c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:04.658563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea434e-b8cf-4022-9580-c31948d3f91c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:04.658894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea434e-b8cf-4022-9580-c31948d3f91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:04.702128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-26ea434e-b8cf-4022-9580-c31948d3f91c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:04.709759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-26ea434e-b8cf-4022-9580-c31948d3f91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:04.875820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 830/1650] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:05.221663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:05.223157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:05.414722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21703bc-cdee-4458-a659-a60440ba35f6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:05.419097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70565d-2be1-4320-acdd-b7ab173caee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:05.419773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70565d-2be1-4320-acdd-b7ab173caee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:05.420060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70565d-2be1-4320-acdd-b7ab173caee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:05.420339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70565d-2be1-4320-acdd-b7ab173caee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:05.442624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70565d-2be1-4320-acdd-b7ab173caee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:05.448037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70565d-2be1-4320-acdd-b7ab173caee3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:05.466408 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 821/1651] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:56:05.581785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8112d-89c9-4d04-a066-36f67deb35f0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:05.582453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8112d-89c9-4d04-a066-36f67deb35f0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:05.582453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8112d-89c9-4d04-a066-36f67deb35f0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:05.584108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 831/1652] 10.222.0.91 () {54 vars in 912 bytes} [Thu May 26 13:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:05.595566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efb56a-5b64-4263-9d6f-e6ee82e91427 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:05.597078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efb56a-5b64-4263-9d6f-e6ee82e91427 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:05.597790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efb56a-5b64-4263-9d6f-e6ee82e91427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:05.679549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-70efb56a-5b64-4263-9d6f-e6ee82e91427 None None] MFA Rules not processed for user `79d624b7de50480bb4002eb23ec2b360`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:05.694647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-70efb56a-5b64-4263-9d6f-e6ee82e91427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:05.729296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 822/1653] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:05.744001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8112d-89c9-4d04-a066-36f67deb35f0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:05.779474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8112d-89c9-4d04-a066-36f67deb35f0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:05.790765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7859ff2-5257-4954-89f3-c2c8b9d25bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:05.793017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7859ff2-5257-4954-89f3-c2c8b9d25bcb service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:05.793017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7859ff2-5257-4954-89f3-c2c8b9d25bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:05.793017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7859ff2-5257-4954-89f3-c2c8b9d25bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:05.819563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7859ff2-5257-4954-89f3-c2c8b9d25bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:05.821030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7859ff2-5257-4954-89f3-c2c8b9d25bcb service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:05.852078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 832/1654] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:07.218351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657d015-21de-4882-aec8-ba6bfece2742 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:07.220680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657d015-21de-4882-aec8-ba6bfece2742 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:07.221883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657d015-21de-4882-aec8-ba6bfece2742 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:07.224747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 823/1655] 10.222.0.91 () {54 vars in 960 bytes} [Thu May 26 13:56:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:07.231800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:07.232317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:07.232817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:07.270719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:07.274281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:07.342508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:07.470228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 833/1656] 10.222.0.91 () {58 vars in 1057 bytes} [Thu May 26 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:07.490593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657d015-21de-4882-aec8-ba6bfece2742 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:07.528406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657d015-21de-4882-aec8-ba6bfece2742 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:07.545925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9744c30-cfb7-456a-b149-bd0143f0de5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:07.546600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9744c30-cfb7-456a-b149-bd0143f0de5d service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:07.546886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9744c30-cfb7-456a-b149-bd0143f0de5d service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:07.547251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9744c30-cfb7-456a-b149-bd0143f0de5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:07.550076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d9744c30-cfb7-456a-b149-bd0143f0de5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:07.580050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:07.582463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:07.603199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9744c30-cfb7-456a-b149-bd0143f0de5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:07.611958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9744c30-cfb7-456a-b149-bd0143f0de5d service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:07.616473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9d8a0-74f9-43cb-b155-5522892d5987 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:07.638943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 824/1657] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:07.642624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:07.645111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:07.646218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:07.647834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] PATH_INFO: `/v3/projects/67bf852b45d8419aad80b70b02e2bf29` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:07.650539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67bf852b45d8419aad80b70b02e2bf29)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:07.654741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:56:07.654741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) May 26 13:56:07.654741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: May 26 13:56:07.657051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:07.889943 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67bf852b45d8419aad80b70b02e2bf29', 'request_id': 'req-85df663a-90b8-432c-95ef-663c47d47e13'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:07.955829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67bf852b45d8419aad80b70b02e2bf29', 'request_id': 'req-85df663a-90b8-432c-95ef-663c47d47e13'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:07.973135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67bf852b45d8419aad80b70b02e2bf29', 'request_id': 'req-85df663a-90b8-432c-95ef-663c47d47e13'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:07.999369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] The token cache is being invalidate because project 67bf852b45d8419aad80b70b02e2bf29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:08.000535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:08.001449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-85df663a-90b8-432c-95ef-663c47d47e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:08.003429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 834/1658] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:07 2022] DELETE /identity/v3/projects/67bf852b45d8419aad80b70b02e2bf29 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:56:08.008697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335ab9e-da62-4b53-8ed6-f48f5166e377 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:08.009005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335ab9e-da62-4b53-8ed6-f48f5166e377 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:08.009315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335ab9e-da62-4b53-8ed6-f48f5166e377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:08.395790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6335ab9e-da62-4b53-8ed6-f48f5166e377 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:08.398850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6335ab9e-da62-4b53-8ed6-f48f5166e377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:08.467864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:08.564564 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 825/1659] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:08.572634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674775f3-1e17-43c1-9db4-4cacb48b934c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:08.575199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674775f3-1e17-43c1-9db4-4cacb48b934c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:08.575199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674775f3-1e17-43c1-9db4-4cacb48b934c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:08.623978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-674775f3-1e17-43c1-9db4-4cacb48b934c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:08.628574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-674775f3-1e17-43c1-9db4-4cacb48b934c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:08.655906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 835/1660] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:08.660014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335ab9e-da62-4b53-8ed6-f48f5166e377 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:08.677023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335ab9e-da62-4b53-8ed6-f48f5166e377 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:08.702677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:08.704028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:08.704343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:08.704622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:08.705380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:08.705686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:08.707475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:08.755600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 826/1661] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:08.760377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674775f3-1e17-43c1-9db4-4cacb48b934c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:08.774822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674775f3-1e17-43c1-9db4-4cacb48b934c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:08.792209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:08.793146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:08.793436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:08.793898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:08.795037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:08.816631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:08.885554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:08.886461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 836/1662] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:08 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:08.899718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53cc4c-b9fc-4cbd-ac2e-ec49083713e3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:09.052644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:09.053365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:09.053648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:09.053921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:09.054622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:09.056545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:09.146515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:09.163219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f43313-e07e-406a-9bd1-b8e4ee7a1dcb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:09.167815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 827/1663] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:08 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:09.198412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:09.199127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:09.199415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:09.209033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:09.213680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:09.222795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:09.250312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:09.255946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 837/1664] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:09.262952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dfb73-ac08-4318-97af-717fbe217681 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:09.282101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8ed2f-f3ac-4ace-b8ce-1fc2a15f3eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:09.283041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8ed2f-f3ac-4ace-b8ce-1fc2a15f3eef admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:09.283451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8ed2f-f3ac-4ace-b8ce-1fc2a15f3eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:09.283729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8ed2f-f3ac-4ace-b8ce-1fc2a15f3eef admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:09.295109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8ed2f-f3ac-4ace-b8ce-1fc2a15f3eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:09.303493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8ed2f-f3ac-4ace-b8ce-1fc2a15f3eef admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:09.341678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 828/1665] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:09.344660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:09.359548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cdc3f-cb70-448a-a66e-18e2fc2aeaf0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:09.380203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:09.381138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:09.381443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:09.381720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] PATH_INFO: `/v3/projects/ae9d999268bf42ab85a7f0954f6b951b/users/f37ef561d90d4d03bef5aba57c274380/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:09.436783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9d999268bf42ab85a7f0954f6b951b, user_id=f37ef561d90d4d03bef5aba57c274380, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:09.440204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:09.492898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 838/1666] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:09 2022] PUT /identity/v3/projects/ae9d999268bf42ab85a7f0954f6b951b/users/f37ef561d90d4d03bef5aba57c274380/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:09.505966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6a053-ea86-426a-95a6-9caa4efb4f91 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:09.506434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6a053-ea86-426a-95a6-9caa4efb4f91 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:09.506515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6a053-ea86-426a-95a6-9caa4efb4f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:09.561423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-02e6a053-ea86-426a-95a6-9caa4efb4f91 None None] MFA Rules not processed for user `f37ef561d90d4d03bef5aba57c274380`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:09.563489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-02e6a053-ea86-426a-95a6-9caa4efb4f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:09.711936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 829/1667] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:09.716201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:09.796241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:09.826679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1930eec-9f69-4d51-b6d3-80540a97e1e9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:09.838257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:09.840881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:09.842296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:09.842296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:09.843120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:09.845744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:09.927399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 839/1668] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:09 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:09.928136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6a053-ea86-426a-95a6-9caa4efb4f91 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:09.943099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6a053-ea86-426a-95a6-9caa4efb4f91 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:10.023727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:10.147956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:10.148669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.148955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.149276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.149989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:10.152401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:10.201187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 830/1669] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:09 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:10.203564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:10.221791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf875d-0d2d-4526-957d-0c3bf97b2b9b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:10.237276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:10.238299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.238774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.239233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.240351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:10.242311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:10.293640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:10.295058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 840/1670] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:10.318973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8365ca-b46d-490e-81b4-ab69f0bc5bac admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:10.329390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:10.330315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.330593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.333104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] PATH_INFO: `/v3/projects/49d9f457f48b4b52af50f5505ae9483f/users/eef43e77addc49fa954d9279acf889fa/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.379468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:10.382775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:10.401937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d9f457f48b4b52af50f5505ae9483f, user_id=eef43e77addc49fa954d9279acf889fa, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:10.415529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:10.418884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e0030-a5a7-4761-bea2-74db9f408cd1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:10.436661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:10.438031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.438903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.439405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] PATH_INFO: `/v3/users/558985b6382c4647b5f734d112f9b5fa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.443166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 831/1671] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:10 2022] PUT /identity/v3/projects/49d9f457f48b4b52af50f5505ae9483f/users/eef43e77addc49fa954d9279acf889fa/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:56:10.449593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=558985b6382c4647b5f734d112f9b5fa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:10.450404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:10.462835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:10.558456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c7c1-7d02-4ed5-8751-e22dcc7fc7f8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:10.566056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:10.567042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.567425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.567798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.568868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:10.570593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:10.608691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 832/1672] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:10.608816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558985b6382c4647b5f734d112f9b5fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:10.609926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:10.631398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa5cd2-093f-45e0-9733-3e4716ce6ad1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:10.638848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3e5f6-1543-40fc-ae9d-af03d7e96642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:10.640092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3e5f6-1543-40fc-ae9d-af03d7e96642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.641096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3e5f6-1543-40fc-ae9d-af03d7e96642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.641670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3e5f6-1543-40fc-ae9d-af03d7e96642 admin admin] PATH_INFO: `/v3/projects/49d9f457f48b4b52af50f5505ae9483f/users/eef43e77addc49fa954d9279acf889fa/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.648391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558985b6382c4647b5f734d112f9b5fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:10.649353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558985b6382c4647b5f734d112f9b5fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:10.651756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558985b6382c4647b5f734d112f9b5fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:10.659109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3e5f6-1543-40fc-ae9d-af03d7e96642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d9f457f48b4b52af50f5505ae9483f, user_id=eef43e77addc49fa954d9279acf889fa, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:10.664931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab26ea6e-ee7c-443a-a710-4ca5c145f3f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558985b6382c4647b5f734d112f9b5fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:10.665906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3e5f6-1543-40fc-ae9d-af03d7e96642 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:10.692421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 833/1673] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:10 2022] PUT /identity/v3/projects/49d9f457f48b4b52af50f5505ae9483f/users/eef43e77addc49fa954d9279acf889fa/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:10.703158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 841/1674] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:10 2022] DELETE /identity/v3/users/558985b6382c4647b5f734d112f9b5fa => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:10.707817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.707975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.708110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.708208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.708392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.708495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:10.743968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:10.745117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:10.780244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] MFA Rules not processed for user `eef43e77addc49fa954d9279acf889fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:10.781596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:10.840720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 842/1675] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:10.854068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:10.857226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:10.913653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:10.951098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf9bc-3ba4-41ca-8c4c-1d19874d8ba5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:10.956651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:10.957691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:10.958243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:10.958861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:11.001029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:11.002995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:11.021753 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:11.031654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 834/1676] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:11.046280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 843/1677] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:11.062382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:11.065665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:11.197370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97cfe-5740-424f-bff8-0584a04399ff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:11.209725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d179-687a-49fb-bee5-da3b3f1b6ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:11.216731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d179-687a-49fb-bee5-da3b3f1b6ccd service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:11.217089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d179-687a-49fb-bee5-da3b3f1b6ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:11.217396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d179-687a-49fb-bee5-da3b3f1b6ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:11.228680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d179-687a-49fb-bee5-da3b3f1b6ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:11.230428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d179-687a-49fb-bee5-da3b3f1b6ccd service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:11.259098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 835/1678] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:11.415229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:11.417314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:11.438787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba6ad3-9287-42d3-96b8-351a14551676 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:11.450327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:11.451009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:11.451393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:11.452065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:11.456983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:11.486338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:11.486338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:11.486542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:11.568843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] MFA Rules not processed for user `c67205b7150c460a81ff545d5ba29cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:11.572991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:11.700169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:11.703018 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:11.742160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 844/1679] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:11.787742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 836/1680] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:11.808695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:11.810229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:11.973110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d40a8f-dc5e-44f7-9bc8-98f36f37d04b service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:11.986434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:11.991065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:11.991608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:11.992513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:11.992800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:11.993082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:12.006319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:12.008002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:12.039967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 845/1681] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:12.056504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38d11a-054f-414b-a22e-e1fe2010fe13 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:12.074635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:12.075889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:12.077616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:12.078076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] PATH_INFO: `/v3/projects/73c3e29dafa248a3bce455dfd308ab7c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:12.087893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c3e29dafa248a3bce455dfd308ab7c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:12.095634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:12.216820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c3e29dafa248a3bce455dfd308ab7c', 'request_id': 'req-705d9795-914b-42a5-8037-c4d2dc6461ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.228976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c3e29dafa248a3bce455dfd308ab7c', 'request_id': 'req-705d9795-914b-42a5-8037-c4d2dc6461ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.240983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c3e29dafa248a3bce455dfd308ab7c', 'request_id': 'req-705d9795-914b-42a5-8037-c4d2dc6461ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.258523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] The token cache is being invalidate because project 73c3e29dafa248a3bce455dfd308ab7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:12.258818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.259493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-705d9795-914b-42a5-8037-c4d2dc6461ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.264196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 837/1682] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:11 2022] DELETE /identity/v3/projects/73c3e29dafa248a3bce455dfd308ab7c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:12.265798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:12.278795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e06f37-51c4-49fd-82f7-290c994be37f service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:12.391620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:12.404416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:12.405364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:12.405658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:12.408487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] PATH_INFO: `/v3/users/d53596339a8a468ea69838eb7605e670` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:12.412758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53596339a8a468ea69838eb7605e670)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:12.418305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:12.482245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53596339a8a468ea69838eb7605e670'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.498280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53596339a8a468ea69838eb7605e670'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.499044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53596339a8a468ea69838eb7605e670'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.499679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53596339a8a468ea69838eb7605e670'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.518086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53596339a8a468ea69838eb7605e670'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:12.550542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 846/1683] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:12 2022] DELETE /identity/v3/users/d53596339a8a468ea69838eb7605e670 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:14.421779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c377e7b-c930-4dcf-bd88-ad3ca6edbdcc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:14.422248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c377e7b-c930-4dcf-bd88-ad3ca6edbdcc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:14.422380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c377e7b-c930-4dcf-bd88-ad3ca6edbdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:14.478574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0c377e7b-c930-4dcf-bd88-ad3ca6edbdcc None None] MFA Rules not processed for user `8d7bcc48c8d844e5b36b1c4e2748e766`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:14.479986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c377e7b-c930-4dcf-bd88-ad3ca6edbdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:14.488684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:14.513818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:14.515296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:14.644971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115fc6-eeb5-4625-b16a-1967a71f4966 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:14.650336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:14.651106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 838/1684] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:14.651669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:14.651960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:14.652237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] PATH_INFO: `/v3/projects/27a095273294426eb873c4dbf544e23f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:14.652629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:14.652928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:14.653240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:14.653815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a095273294426eb873c4dbf544e23f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:14.655794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:14.698228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] MFA Rules not processed for user `8d7bcc48c8d844e5b36b1c4e2748e766`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:14.699736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:14.711954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 839/1685] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:14.721739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:14.724271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:14.738729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a095273294426eb873c4dbf544e23f', 'request_id': 'req-25a2e730-3f90-4c58-8238-bfaefbad5e2b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:14.764166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a095273294426eb873c4dbf544e23f', 'request_id': 'req-25a2e730-3f90-4c58-8238-bfaefbad5e2b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:14.777130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a095273294426eb873c4dbf544e23f', 'request_id': 'req-25a2e730-3f90-4c58-8238-bfaefbad5e2b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:14.815755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] The token cache is being invalidate because project 27a095273294426eb873c4dbf544e23f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:14.816062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:14.817586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25a2e730-3f90-4c58-8238-bfaefbad5e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:14.827697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 847/1686] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:14 2022] DELETE /identity/v3/projects/27a095273294426eb873c4dbf544e23f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:56:14.831602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8309af-10ed-49ea-b7d6-732031f85bef None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:14.831896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8309af-10ed-49ea-b7d6-732031f85bef None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:14.832177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8309af-10ed-49ea-b7d6-732031f85bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:14.840091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757021d-f7b4-43fe-b52a-bd96f9907f69 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:14.864366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:14.881762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7e8309af-10ed-49ea-b7d6-732031f85bef None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:14.885778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7e8309af-10ed-49ea-b7d6-732031f85bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:15.011863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:15.012838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.015011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.015011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.021476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:15.076265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 848/1687] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:15.089067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36788a82-8071-4e50-b5ab-7c43d0bf2f9a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.089299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36788a82-8071-4e50-b5ab-7c43d0bf2f9a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.089437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36788a82-8071-4e50-b5ab-7c43d0bf2f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.105512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:15.121626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-36788a82-8071-4e50-b5ab-7c43d0bf2f9a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:15.122863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-36788a82-8071-4e50-b5ab-7c43d0bf2f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:15.142562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:15.144447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:15.150188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 849/1688] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:15.154446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36788a82-8071-4e50-b5ab-7c43d0bf2f9a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:15.160382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 840/1689] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:15.185982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36788a82-8071-4e50-b5ab-7c43d0bf2f9a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:15.211737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:15.212444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.212728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.213010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.214044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:15.214359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:15.216074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:15.250690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 850/1690] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:15.256998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:15.278952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50859e5e-359b-4b97-afca-3295f1ca60ab service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:15.286606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:15.287832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.288296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.288584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.289374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:15.292627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:15.363224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 841/1691] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:15 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:15.364221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:15.380173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c281-42d8-4c5b-8cae-db9710f9114c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:15.479311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:15.480011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.480297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.480579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.481339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:15.484213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:15.534243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:15.553294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 851/1692] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:15 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:15.560501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69320dc-ca72-456e-b48a-637381d029f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:15.575768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:15.576762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.577052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.577358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.578144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:15.579785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:15.611013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 842/1693] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:15.615744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:15.642711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694bc43-6a5a-4957-8b9f-d66057527bc8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:15.649592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2b9b5-248b-4846-86b6-bce6c0a94c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:15.650726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b9b5-248b-4846-86b6-bce6c0a94c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.651085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b9b5-248b-4846-86b6-bce6c0a94c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.651371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b9b5-248b-4846-86b6-bce6c0a94c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.652173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2b9b5-248b-4846-86b6-bce6c0a94c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:15.654817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2b9b5-248b-4846-86b6-bce6c0a94c27 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:15.674673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:15.682153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 852/1694] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:15.691538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb741093-147b-498a-a4b8-7abc36ecf742 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:15.699468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac4957-cdee-4738-89fc-0055fb3eb440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:15.704748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac4957-cdee-4738-89fc-0055fb3eb440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.705121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac4957-cdee-4738-89fc-0055fb3eb440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.706193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac4957-cdee-4738-89fc-0055fb3eb440 admin admin] PATH_INFO: `/v3/projects/bad238a7ea3e461f985a3239653b8a93/users/313502cc8ba54ea5a0d67962e99e6d2f/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.751128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac4957-cdee-4738-89fc-0055fb3eb440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad238a7ea3e461f985a3239653b8a93, user_id=313502cc8ba54ea5a0d67962e99e6d2f, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:15.754024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac4957-cdee-4738-89fc-0055fb3eb440 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:15.790657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 843/1695] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:15 2022] PUT /identity/v3/projects/bad238a7ea3e461f985a3239653b8a93/users/313502cc8ba54ea5a0d67962e99e6d2f/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:15.801996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049670-3564-4c35-b844-a09b9ff0cce3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.802179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049670-3564-4c35-b844-a09b9ff0cce3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.802501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049670-3564-4c35-b844-a09b9ff0cce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.842603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2c049670-3564-4c35-b844-a09b9ff0cce3 None None] MFA Rules not processed for user `313502cc8ba54ea5a0d67962e99e6d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:15.844371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2c049670-3564-4c35-b844-a09b9ff0cce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:15.871101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628db963-d742-4b57-b19a-0c14b178b699 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.871101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628db963-d742-4b57-b19a-0c14b178b699 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.871101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628db963-d742-4b57-b19a-0c14b178b699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.938580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-628db963-d742-4b57-b19a-0c14b178b699 None None] MFA Rules not processed for user `592975b43454407abde6c7cab2504982`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:15.939820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-628db963-d742-4b57-b19a-0c14b178b699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:15.962575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:15.982971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 853/1696] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:15.985635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592b47d-4b10-4940-b57e-3f9fb498b63a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:15.986382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592b47d-4b10-4940-b57e-3f9fb498b63a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:15.986978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592b47d-4b10-4940-b57e-3f9fb498b63a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:15.988770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 854/1697] 10.222.0.91 () {54 vars in 912 bytes} [Thu May 26 13:56:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:15.989683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592b47d-4b10-4940-b57e-3f9fb498b63a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:16.084755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 844/1698] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:16.086063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592b47d-4b10-4940-b57e-3f9fb498b63a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:16.086277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e7df-410f-4765-bd01-6bec9bbbf8bf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.086570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e7df-410f-4765-bd01-6bec9bbbf8bf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.086849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e7df-410f-4765-bd01-6bec9bbbf8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.104212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:16.105100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.105519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.105519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.106250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:16.108208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:16.110840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-13c2e7df-410f-4765-bd01-6bec9bbbf8bf None None] MFA Rules not processed for user `79d624b7de50480bb4002eb23ec2b360`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:16.112045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-13c2e7df-410f-4765-bd01-6bec9bbbf8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:16.168485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 855/1699] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:15 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:16.169589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:16.174140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:16.211607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 845/1700] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:56:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:16.213294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2e7df-410f-4765-bd01-6bec9bbbf8bf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:16.241196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2e7df-410f-4765-bd01-6bec9bbbf8bf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:16.286655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:16.298926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feb97b-d35d-4733-9987-fca258eb07f4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:16.307252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:16.308163 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.308601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.308870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.374756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:16.376663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.377145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.379957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.382384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:16.385129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:16.390041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:16.391850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:16.405919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 856/1701] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:16.406938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:16.408436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:16.435306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:16.452515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 846/1702] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:16 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:16.458000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:16.505761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093d483-2f6d-4d47-a6ed-b308a89bd6ad service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:16.515488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:16.518176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376743-0f92-4ed9-9e9b-f1afc15863af admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:16.519086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.521674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.522089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.540115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:16.543211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:16.548476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:16.550147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.550744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.551035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.551804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:16.561568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:16.593140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 857/1703] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:16.613919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 847/1704] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:16.614101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:16.639581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad5cec-8bea-4c34-ae02-ea2e71491c4d service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:16.645445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:16.646365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.646641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.647002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] PATH_INFO: `/v3/projects/48800ade8cfa4a27aa94e82613cc48b7/users/814ede463e5048beac310edf48ecd107/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.668573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48800ade8cfa4a27aa94e82613cc48b7, user_id=814ede463e5048beac310edf48ecd107, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:16.679567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:16.716187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 858/1705] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:16 2022] PUT /identity/v3/projects/48800ade8cfa4a27aa94e82613cc48b7/users/814ede463e5048beac310edf48ecd107/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:16.721689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:16.738253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79cd98-5206-4da0-a28d-9dd56889e4ca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:16.983398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0948df9-6e58-4557-97f8-17317b71b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:16.985021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0948df9-6e58-4557-97f8-17317b71b335 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:16.985346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0948df9-6e58-4557-97f8-17317b71b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:16.985628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0948df9-6e58-4557-97f8-17317b71b335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:16.986366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0948df9-6e58-4557-97f8-17317b71b335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:16.987982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0948df9-6e58-4557-97f8-17317b71b335 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:17.030490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 848/1706] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:17.034367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:17.051868 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c25122-51bb-426e-a26c-4853bea2c373 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:17.074595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaef335-23eb-498c-9c3c-141eeea8039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:17.075398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaef335-23eb-498c-9c3c-141eeea8039d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:17.075716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaef335-23eb-498c-9c3c-141eeea8039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:17.075968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaef335-23eb-498c-9c3c-141eeea8039d admin admin] PATH_INFO: `/v3/projects/48800ade8cfa4a27aa94e82613cc48b7/users/814ede463e5048beac310edf48ecd107/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:17.078588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaef335-23eb-498c-9c3c-141eeea8039d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48800ade8cfa4a27aa94e82613cc48b7, user_id=814ede463e5048beac310edf48ecd107, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:17.081439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaef335-23eb-498c-9c3c-141eeea8039d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:17.110203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 859/1707] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:17 2022] PUT /identity/v3/projects/48800ade8cfa4a27aa94e82613cc48b7/users/814ede463e5048beac310edf48ecd107/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:17.121626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:17.121925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:17.122208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:17.211714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] MFA Rules not processed for user `814ede463e5048beac310edf48ecd107`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:17.213983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:17.348982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:17.349305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:17.349707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:17.407603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:17.409048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:17.444033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 849/1708] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:17.463487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:17.466098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:17.494012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:17.595665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a0dc0-4376-4aec-95ad-0cc0eeb61301 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:17.604311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f168829-f5ad-4c0d-8d93-4e228cee8bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:17.605261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168829-f5ad-4c0d-8d93-4e228cee8bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:17.607963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168829-f5ad-4c0d-8d93-4e228cee8bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:17.608246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168829-f5ad-4c0d-8d93-4e228cee8bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:17.613969 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 860/1709] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:17.689566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f168829-f5ad-4c0d-8d93-4e228cee8bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:17.692269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f168829-f5ad-4c0d-8d93-4e228cee8bc1 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:17.729989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 850/1710] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:17.915213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:17.917543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:18.118561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0085026-f761-401e-9fb0-0f3c9a923dca None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:18.158443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:18.159375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:18.159890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:18.160365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:18.198657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:18.202654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:18.238618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 861/1711] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 13:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:56:18.753672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24406a-7ab3-4442-81b8-c5e8542415c9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:18.755081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24406a-7ab3-4442-81b8-c5e8542415c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:18.756150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24406a-7ab3-4442-81b8-c5e8542415c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:18.801779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6b24406a-7ab3-4442-81b8-c5e8542415c9 None None] MFA Rules not processed for user `8d7bcc48c8d844e5b36b1c4e2748e766`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:18.810138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b24406a-7ab3-4442-81b8-c5e8542415c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:19.028196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 851/1712] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:19.686329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:19.712983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd7fd-3e3c-4e83-8a35-61882d2eb8f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:19.713308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd7fd-3e3c-4e83-8a35-61882d2eb8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:19.716221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd7fd-3e3c-4e83-8a35-61882d2eb8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:19.755974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:19.804148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-646dd7fd-3e3c-4e83-8a35-61882d2eb8f6 None None] MFA Rules not processed for user `c67205b7150c460a81ff545d5ba29cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:19.812046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-646dd7fd-3e3c-4e83-8a35-61882d2eb8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:19.868134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c841f9-9535-4b53-bfe1-40b572ba1d49 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:19.876782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:19.877814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:19.878103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:19.878385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] PATH_INFO: `/v3/users/313502cc8ba54ea5a0d67962e99e6d2f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:19.882105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=313502cc8ba54ea5a0d67962e99e6d2f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:19.898114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:20.025014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 852/1713] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:20.049001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313502cc8ba54ea5a0d67962e99e6d2f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:20.089191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313502cc8ba54ea5a0d67962e99e6d2f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:20.098038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313502cc8ba54ea5a0d67962e99e6d2f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:20.106758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313502cc8ba54ea5a0d67962e99e6d2f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:20.143834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313502cc8ba54ea5a0d67962e99e6d2f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:20.192111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:20.192415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:20.192700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:20.195016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 862/1714] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:19 2022] DELETE /identity/v3/users/313502cc8ba54ea5a0d67962e99e6d2f => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:20.292599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:20.298498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:20.597635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 853/1715] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:20.732593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:20.733896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:20.932845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146221f6-d4bc-4eb8-9507-2e06627aa659 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:20.936837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf328a2-976b-44b8-a817-00deba48afb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:20.937976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf328a2-976b-44b8-a817-00deba48afb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:20.938374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf328a2-976b-44b8-a817-00deba48afb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:20.938732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf328a2-976b-44b8-a817-00deba48afb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:20.965133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf328a2-976b-44b8-a817-00deba48afb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:20.966763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf328a2-976b-44b8-a817-00deba48afb1 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:20.979548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 863/1716] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:22.491059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:22.492826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:22.562874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a275cb2-9a6b-442d-a499-9e50a5e40bc1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:22.575451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:22.577055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:22.578720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:22.580317 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] PATH_INFO: `/v3/projects/bad238a7ea3e461f985a3239653b8a93` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:22.587378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad238a7ea3e461f985a3239653b8a93)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:22.599668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:22.605408 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:22.613510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697ef9-66c3-4faf-88dc-1dd811bedd76 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:22.614008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697ef9-66c3-4faf-88dc-1dd811bedd76 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:22.616148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697ef9-66c3-4faf-88dc-1dd811bedd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:22.650170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-89697ef9-66c3-4faf-88dc-1dd811bedd76 None None] MFA Rules not processed for user `592975b43454407abde6c7cab2504982`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:22.658733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-89697ef9-66c3-4faf-88dc-1dd811bedd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:22.752336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad238a7ea3e461f985a3239653b8a93', 'request_id': 'req-283c0b38-4332-46eb-b745-9b265a1d214a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:22.768584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad238a7ea3e461f985a3239653b8a93', 'request_id': 'req-283c0b38-4332-46eb-b745-9b265a1d214a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:22.801794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad238a7ea3e461f985a3239653b8a93', 'request_id': 'req-283c0b38-4332-46eb-b745-9b265a1d214a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:22.830522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] The token cache is being invalidate because project bad238a7ea3e461f985a3239653b8a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:22.830741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:22.830836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:22.831896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:22.837275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 854/1717] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:22 2022] DELETE /identity/v3/projects/bad238a7ea3e461f985a3239653b8a93 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:22.841131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:22.842790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:22.927821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 864/1718] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:23.100628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c0b38-4332-46eb-b745-9b265a1d214a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:23.128824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:23.133865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:23.137191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:23.137608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] PATH_INFO: `/v3/users/814ede463e5048beac310edf48ecd107` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:23.142018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=814ede463e5048beac310edf48ecd107)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:23.150773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:23.337106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814ede463e5048beac310edf48ecd107'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:23.360714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814ede463e5048beac310edf48ecd107'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:23.392404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814ede463e5048beac310edf48ecd107'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:23.393530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814ede463e5048beac310edf48ecd107'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:23.394311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5c36ffc-e31f-4e23-bbe7-daa590549357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814ede463e5048beac310edf48ecd107'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:23.418255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 855/1719] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:22 2022] DELETE /identity/v3/users/814ede463e5048beac310edf48ecd107 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:24.514828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89697ef9-66c3-4faf-88dc-1dd811bedd76 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:24.535244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89697ef9-66c3-4faf-88dc-1dd811bedd76 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:24.697889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:24.698784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:24.699102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:24.699374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] PATH_INFO: `/v3/projects/48800ade8cfa4a27aa94e82613cc48b7` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:24.706070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48800ade8cfa4a27aa94e82613cc48b7)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:24.708662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:24.815531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48800ade8cfa4a27aa94e82613cc48b7', 'request_id': 'req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:24.822675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48800ade8cfa4a27aa94e82613cc48b7', 'request_id': 'req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:24.829567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48800ade8cfa4a27aa94e82613cc48b7', 'request_id': 'req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:24.850266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] The token cache is being invalidate because project 48800ade8cfa4a27aa94e82613cc48b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:24.850790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:24.851667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-52a9ac71-2ad2-430f-844e-dc8a42ceeda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:24.859693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 865/1720] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:24 2022] DELETE /identity/v3/projects/48800ade8cfa4a27aa94e82613cc48b7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:24.860837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:24.861339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:24.861617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:24.920020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:24.924643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:24.974051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:25.040513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475f78-0a0b-402f-a625-21da18cc4ed6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:25.040802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475f78-0a0b-402f-a625-21da18cc4ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:25.041145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475f78-0a0b-402f-a625-21da18cc4ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:25.103716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-04475f78-0a0b-402f-a625-21da18cc4ed6 None None] MFA Rules not processed for user `79d624b7de50480bb4002eb23ec2b360`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:25.106455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-04475f78-0a0b-402f-a625-21da18cc4ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:25.274474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 856/1721] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:25.277650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:25.289686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:25.353332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051efaf-b7a3-4c2f-b08e-1c0fa05c58be None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:25.367492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 866/1722] 10.222.0.82 () {58 vars in 1015 bytes} [Thu May 26 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:25.381183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:25.391126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:25.391126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:25.392186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:25.392186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:25.392186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:25.392186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] PATH_INFO: `/v3/users/4ab0abfc9bb24cbebc5c04adcec47c10` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:25.401626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab0abfc9bb24cbebc5c04adcec47c10)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:25.407218 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:25.464682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:25.471654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:25.508791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 867/1723] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:25.509802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:25.512869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:25.578414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eefe0-ce2c-4bb0-84fd-60895724809a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:25.589375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:25.591110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab0abfc9bb24cbebc5c04adcec47c10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:25.595931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:25.596263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:25.596568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:25.622792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:25.624933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab0abfc9bb24cbebc5c04adcec47c10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:25.630168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:25.663241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab0abfc9bb24cbebc5c04adcec47c10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:25.666078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab0abfc9bb24cbebc5c04adcec47c10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:25.668135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a039c119-6f18-4c63-a041-4c220cbd21da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab0abfc9bb24cbebc5c04adcec47c10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:25.674145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 868/1724] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:25.676768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:25.703622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:25.713502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354af23-01de-4b21-9c2c-39abb2441791 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:25.717487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 857/1725] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:25 2022] DELETE /identity/v3/users/4ab0abfc9bb24cbebc5c04adcec47c10 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:25.731383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:25.732300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:25.732588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:25.862242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:25.863549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:26.016454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:26.018599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:26.019751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:26.020237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:26.022869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:26.023449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:26.029108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:26.062040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 869/1726] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:26.063756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:26.069082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:26.084740 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a857fb-8c08-4c6a-ae63-12bf36430214 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:26.098179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:26.100224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:26.100224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:26.102341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] PATH_INFO: `/v3/users/34e66e352264450b9f99e2f710e6ba91` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:26.109080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e66e352264450b9f99e2f710e6ba91)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:26.124339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 858/1727] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:26.127972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:26.128133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:26.142080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:26.164836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a53e7f-bad8-4b3c-94a9-1e9a71d5399b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:26.169949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:26.170835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:26.171125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:26.171596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:26.172453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:26.174586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:26.300880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 859/1728] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:26 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:26.302649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:26.306673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:26.329366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:26.333024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e66e352264450b9f99e2f710e6ba91'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:26.383857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e66e352264450b9f99e2f710e6ba91'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:26.391515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e66e352264450b9f99e2f710e6ba91'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:26.394317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e66e352264450b9f99e2f710e6ba91'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:26.449921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e66e352264450b9f99e2f710e6ba91'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:26.497274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 870/1729] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:26 2022] DELETE /identity/v3/users/34e66e352264450b9f99e2f710e6ba91 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:26.498507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:26.500945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:26.518038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5e236-b9cf-4b59-967d-15a858e25359 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:26.695947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ba42-bf18-4781-b15a-184c767c4bc1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:26.729628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:26.732125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:26.732511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09aff08-26eb-46d3-95e6-45fe5ff54923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:26.733634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aff08-26eb-46d3-95e6-45fe5ff54923 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:26.734634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aff08-26eb-46d3-95e6-45fe5ff54923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:26.734932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:26.735042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09aff08-26eb-46d3-95e6-45fe5ff54923 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:26.735491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:26.736123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09aff08-26eb-46d3-95e6-45fe5ff54923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:26.743665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:26.753652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09aff08-26eb-46d3-95e6-45fe5ff54923 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:26.811875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:26.818151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:26.837571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:26.854245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 871/1730] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:26 2022] POST /identity/v3/users => generated 291 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:26.855521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:26.855820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:26.856102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:26.865494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 860/1731] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:26.866761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:26.889102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a7488-49c8-4dfd-b980-934a18cc8fcb service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:26.896590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:26.897476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:26.897836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:26.898115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:26.898848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:26.900955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:26.925879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 861/1732] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:26.933038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:26.937820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:26.941280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:26.984647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 872/1733] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:26.992275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bb4bb-8d69-48c7-a3db-402ea7592b07 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:27.006023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:27.007576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:27.008069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:27.008491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:27.009876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:27.012994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:27.036617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:27.038613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:27.064575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e1d0-632c-49ce-8c09-67b6ab625ea8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:27.072042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 862/1734] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:27.073238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:27.073502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd18e8-d0c8-4cf8-aabc-9ceee0c07322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:27.074284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd18e8-d0c8-4cf8-aabc-9ceee0c07322 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:27.074932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd18e8-d0c8-4cf8-aabc-9ceee0c07322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:27.075903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd18e8-d0c8-4cf8-aabc-9ceee0c07322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:27.101262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ee78-2338-41c3-9bc9-caf9bdbd1e83 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:27.104945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd18e8-d0c8-4cf8-aabc-9ceee0c07322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:27.108616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd18e8-d0c8-4cf8-aabc-9ceee0c07322 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:27.114465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:27.126844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 873/1735] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:56:27.131863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:27.136683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:27.137678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] PATH_INFO: `/v3/projects/a387c957936242528b0b3a90a1059586/users/27da27ab572b4cfba4d547760ad62e5d/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:27.186320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a387c957936242528b0b3a90a1059586, user_id=27da27ab572b4cfba4d547760ad62e5d, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:27.194069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:27.232972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 863/1736] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:27 2022] PUT /identity/v3/projects/a387c957936242528b0b3a90a1059586/users/27da27ab572b4cfba4d547760ad62e5d/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:27.243871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76f92-a983-4975-824d-a1fe536b4d54 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:27.244171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76f92-a983-4975-824d-a1fe536b4d54 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:27.244438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76f92-a983-4975-824d-a1fe536b4d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:27.308013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b8c76f92-a983-4975-824d-a1fe536b4d54 None None] MFA Rules not processed for user `27da27ab572b4cfba4d547760ad62e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:27.312061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b8c76f92-a983-4975-824d-a1fe536b4d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:27.544806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 874/1737] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:27.554364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:27.789021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e8c9-a3ae-4f55-be08-dbcca93d667f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:27.799462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3d344-e599-4a22-81fb-b951ef60f0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:27.800179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3d344-e599-4a22-81fb-b951ef60f0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:27.800467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3d344-e599-4a22-81fb-b951ef60f0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:27.800781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3d344-e599-4a22-81fb-b951ef60f0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:27.805135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3d344-e599-4a22-81fb-b951ef60f0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:27.807301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3d344-e599-4a22-81fb-b951ef60f0d7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:27.868023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:27.875525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 864/1738] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:27 2022] POST /identity/v3/projects => generated 381 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:27.879741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c76f92-a983-4975-824d-a1fe536b4d54 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:27.893873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c76f92-a983-4975-824d-a1fe536b4d54 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:28.184273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:28.185617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:28.193624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:28.194155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:28.195059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:28.206273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:28.230842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313887d2-af49-4c7c-a702-56a8637a558e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:28.231019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313887d2-af49-4c7c-a702-56a8637a558e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:28.231302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313887d2-af49-4c7c-a702-56a8637a558e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:28.251946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 865/1739] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:56:28 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:28.292958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313887d2-af49-4c7c-a702-56a8637a558e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:28.295471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 875/1740] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:27 2022] POST /identity/v3/users => generated 297 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:28.316508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313887d2-af49-4c7c-a702-56a8637a558e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:28.336685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e01c5-8581-48d6-aa2f-2f10f7e5b91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:28.348511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e01c5-8581-48d6-aa2f-2f10f7e5b91b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:28.348857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e01c5-8581-48d6-aa2f-2f10f7e5b91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:28.349139 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e01c5-8581-48d6-aa2f-2f10f7e5b91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:28.349957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e01c5-8581-48d6-aa2f-2f10f7e5b91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:28.351602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e01c5-8581-48d6-aa2f-2f10f7e5b91b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:28.400141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:28.415256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 866/1741] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:56:28.417899 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486ebb-4f36-4195-bf7b-60345a063888 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:28.432807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:28.433940 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:28.434235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:28.434546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] PATH_INFO: `/v3/projects/7c9b3fc9d01a41f992304fa82153c9e0/users/c766af119e1146d9ac3860d880f699e2/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:28.511960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9b3fc9d01a41f992304fa82153c9e0, user_id=c766af119e1146d9ac3860d880f699e2, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:28.514709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884d6a-d200-4709-8f73-800892059198 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:28.515520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884d6a-d200-4709-8f73-800892059198 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:28.515843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884d6a-d200-4709-8f73-800892059198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:28.525799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:28.579519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 876/1742] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:28 2022] PUT /identity/v3/projects/7c9b3fc9d01a41f992304fa82153c9e0/users/c766af119e1146d9ac3860d880f699e2/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:56:28.586144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:28.587113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8e884d6a-d200-4709-8f73-800892059198 None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:28.588550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e884d6a-d200-4709-8f73-800892059198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:28.602133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:28.769536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:28.870530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 867/1743] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:56:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:28.871576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e884d6a-d200-4709-8f73-800892059198 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:28.888527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e7e57-b378-464d-8956-801d3434530b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:28.896115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e884d6a-d200-4709-8f73-800892059198 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:28.913867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:28.914713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:28.915117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:28.915499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:28.916350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:28.918122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:28.919255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:28.921382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:28.921822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:28.922210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] PATH_INFO: `/v3/projects/74e1eca47d504eb887f1c9813f537010` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:28.925820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e1eca47d504eb887f1c9813f537010)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:28.928242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:28.960493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 868/1744] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:28.962253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:28.965837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:29.007255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db324e2-84a0-4c44-b1f4-65c153aa29c5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:29.013328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:29.014759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:29.016455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:29.016962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] PATH_INFO: `/v3/projects/dadb5f6b57c04d5d8cdf41a03824eb3b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:29.018542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dadb5f6b57c04d5d8cdf41a03824eb3b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:29.045605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:29.105307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e1eca47d504eb887f1c9813f537010', 'request_id': 'req-6ff919c6-2bff-4826-ada1-00c54a9316fe'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.140411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e1eca47d504eb887f1c9813f537010', 'request_id': 'req-6ff919c6-2bff-4826-ada1-00c54a9316fe'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.155944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e1eca47d504eb887f1c9813f537010', 'request_id': 'req-6ff919c6-2bff-4826-ada1-00c54a9316fe'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.224075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] The token cache is being invalidate because project 74e1eca47d504eb887f1c9813f537010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:29.224420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.225180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.235837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 877/1745] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:28 2022] DELETE /identity/v3/projects/74e1eca47d504eb887f1c9813f537010 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:29.237200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:29.242095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:29.278429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dadb5f6b57c04d5d8cdf41a03824eb3b', 'request_id': 'req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.317716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dadb5f6b57c04d5d8cdf41a03824eb3b', 'request_id': 'req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.338745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dadb5f6b57c04d5d8cdf41a03824eb3b', 'request_id': 'req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.363519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] The token cache is being invalidate because project dadb5f6b57c04d5d8cdf41a03824eb3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:29.364090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.365399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:29.375517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 869/1746] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:28 2022] DELETE /identity/v3/projects/dadb5f6b57c04d5d8cdf41a03824eb3b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:29.375517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:29.381199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:29.496497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff919c6-2bff-4826-ada1-00c54a9316fe admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:29.601483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:29.636182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37c0e-7074-4c48-a4fd-3cc696d624fc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:29.651465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:29.652403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:29.652687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:29.654950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] PATH_INFO: `/v3/projects/7c9b3fc9d01a41f992304fa82153c9e0/users/c766af119e1146d9ac3860d880f699e2/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:29.659134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9b3fc9d01a41f992304fa82153c9e0, user_id=c766af119e1146d9ac3860d880f699e2, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:29.661875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:29.714013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:29.715701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 870/1747] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:29 2022] PUT /identity/v3/projects/7c9b3fc9d01a41f992304fa82153c9e0/users/c766af119e1146d9ac3860d880f699e2/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:29.717650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:29.723811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:29.740200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae1171-2039-4e4f-be08-1104b70ee950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:29.741956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1171-2039-4e4f-be08-1104b70ee950 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:29.742926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1171-2039-4e4f-be08-1104b70ee950 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:29.743222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1171-2039-4e4f-be08-1104b70ee950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:29.753729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bbae1171-2039-4e4f-be08-1104b70ee950 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:29.775339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae1171-2039-4e4f-be08-1104b70ee950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:29.777427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae1171-2039-4e4f-be08-1104b70ee950 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:29.796670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 878/1748] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:29.798264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591d43-bc49-4b7e-b84b-0aa2e0935ade None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:29.798555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591d43-bc49-4b7e-b84b-0aa2e0935ade None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:29.799090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591d43-bc49-4b7e-b84b-0aa2e0935ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:29.822987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d1591d43-bc49-4b7e-b84b-0aa2e0935ade None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:29.825950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d1591d43-bc49-4b7e-b84b-0aa2e0935ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:29.938200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d99d40-ffec-4993-86c1-547ae7c07ff2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:29.954574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:29.955483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:29.955769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:29.956043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] PATH_INFO: `/v3/users/23a4247590974d0a8ef02afd828edd26` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:29.961687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a4247590974d0a8ef02afd828edd26)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:29.966689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:30.026414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 879/1749] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:30.034362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:30.041929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:30.044429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:30.103016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a4247590974d0a8ef02afd828edd26'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:30.116434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a4247590974d0a8ef02afd828edd26'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:30.140359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] MFA Rules not processed for user `c766af119e1146d9ac3860d880f699e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:30.141624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:30.144362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a4247590974d0a8ef02afd828edd26'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:30.145484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a4247590974d0a8ef02afd828edd26'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:30.146336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da94126a-e605-40df-abff-dd33b9c14500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a4247590974d0a8ef02afd828edd26'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:30.173966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 871/1750] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:29 2022] DELETE /identity/v3/users/23a4247590974d0a8ef02afd828edd26 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:30.179540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130a761-80d4-429e-9913-08e263e407ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:30.179540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130a761-80d4-429e-9913-08e263e407ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:30.179540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130a761-80d4-429e-9913-08e263e407ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:30.208435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b130a761-80d4-429e-9913-08e263e407ab None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:30.211502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b130a761-80d4-429e-9913-08e263e407ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:30.353196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 880/1751] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:30.368768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:30.370475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:30.406217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 872/1752] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:30.415530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130a761-80d4-429e-9913-08e263e407ab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:30.447622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130a761-80d4-429e-9913-08e263e407ab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:30.458721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:30.459470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:30.459767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:30.460051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:30.461534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:30.461534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:30.463471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:30.465264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:30.493681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 873/1753] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:30.494669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:30.538079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0d559-e430-416d-9bdc-5a95cea767bd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:30.579209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:30.581441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:30.585592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:30.586174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:30.589257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:30.597920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:30.652755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257701ca-d51f-4d89-a19f-d30201f7b303 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:30.712828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 874/1754] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:30 2022] POST /identity/v3/projects => generated 405 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:30.719501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:30.776861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:30.849706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:30.850378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:30.850667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:30.850942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:30.938877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ee49d-dee0-420e-b4f6-fdc4a093ca58 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:30.951209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:30.951977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:30.953341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:30.953629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:30.954602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:30.961749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:31.040698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 875/1755] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:30 2022] POST /identity/v3/users => generated 303 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:31.042765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:31.076162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:31.078268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:31.079279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713fea0-ad7e-48df-983f-ab15edfce231 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:31.099516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:31.100241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:31.103221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:31.103221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:31.103221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:31.104967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:31.114213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 881/1756] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 747 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:31.126520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 876/1757] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:31.127867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:31.144223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c2d1a-e2df-467b-86e4-01bde50d1282 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:31.160385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d33a33-c1a8-4b05-ae00-550506146e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:31.161455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d33a33-c1a8-4b05-ae00-550506146e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:31.163161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d33a33-c1a8-4b05-ae00-550506146e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:31.163629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d33a33-c1a8-4b05-ae00-550506146e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:31.165010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d33a33-c1a8-4b05-ae00-550506146e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:31.170235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d33a33-c1a8-4b05-ae00-550506146e60 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:31.210300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 882/1758] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:31.212220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:31.264620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c6724-addb-4bf9-98ad-452dcb2a8152 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:31.290729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:31.291563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:31.291856 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:31.292130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] PATH_INFO: `/v3/projects/55c936bd8e71491da96a2abe726918a2/users/3b0ba1e92d8e4663b8bb77bc90e80cda/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:31.393744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c936bd8e71491da96a2abe726918a2, user_id=3b0ba1e92d8e4663b8bb77bc90e80cda, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:31.397395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:31.459177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 877/1759] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:31 2022] PUT /identity/v3/projects/55c936bd8e71491da96a2abe726918a2/users/3b0ba1e92d8e4663b8bb77bc90e80cda/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:31.461310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:31.461773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:31.462216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:31.514237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:31.516043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:31.687521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 883/1760] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:31.854776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:31.856950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:32.012134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d84495-e216-47a2-a226-43f9551dc5e9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:32.031987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650559-18cc-438a-9da1-bbf3ba763ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:32.033132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650559-18cc-438a-9da1-bbf3ba763ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:32.033669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650559-18cc-438a-9da1-bbf3ba763ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:32.034165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650559-18cc-438a-9da1-bbf3ba763ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:32.052852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8650559-18cc-438a-9da1-bbf3ba763ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:32.055097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8650559-18cc-438a-9da1-bbf3ba763ea4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:32.077125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:32.079606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:32.089570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 878/1761] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:32.107761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd0842-6549-482a-9683-7b7a6bd4268e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:32.118970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:32.120583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:32.121018 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:32.121511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] PATH_INFO: `/v3/users/27da27ab572b4cfba4d547760ad62e5d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:32.122996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27da27ab572b4cfba4d547760ad62e5d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:32.126104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:32.524310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27da27ab572b4cfba4d547760ad62e5d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:32.551485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27da27ab572b4cfba4d547760ad62e5d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:32.552280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27da27ab572b4cfba4d547760ad62e5d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:32.553583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27da27ab572b4cfba4d547760ad62e5d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:32.556466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:32.579484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27da27ab572b4cfba4d547760ad62e5d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:32.607074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 884/1762] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:32 2022] DELETE /identity/v3/users/27da27ab572b4cfba4d547760ad62e5d => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:32.612796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:32.613245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:32.613533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:32.654654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:32.656665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:32.706628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:32.754747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 879/1763] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:32.857258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:32.858920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:33.045633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be4124-aa91-4d94-8ed4-6abf2662e983 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:33.055097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce7224-f873-46b1-ae54-1d51b108147c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:33.056010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce7224-f873-46b1-ae54-1d51b108147c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:33.056366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce7224-f873-46b1-ae54-1d51b108147c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:33.056733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce7224-f873-46b1-ae54-1d51b108147c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:33.074929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce7224-f873-46b1-ae54-1d51b108147c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:33.078085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce7224-f873-46b1-ae54-1d51b108147c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:33.120219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 885/1764] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:33.307909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:33.309553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:33.326492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291be1-ece0-42d2-b6bc-39099f7d988a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:33.347177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:33.348068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:33.348529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:33.348963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] PATH_INFO: `/v3/projects/f223017dc9f7489a8089463f44fd53a9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:33.350281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f223017dc9f7489a8089463f44fd53a9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:33.352174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:33.440306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f223017dc9f7489a8089463f44fd53a9', 'request_id': 'req-b0b2a328-97ef-48bd-81eb-79936cbcca04'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:33.471360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f223017dc9f7489a8089463f44fd53a9', 'request_id': 'req-b0b2a328-97ef-48bd-81eb-79936cbcca04'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:33.486027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f223017dc9f7489a8089463f44fd53a9', 'request_id': 'req-b0b2a328-97ef-48bd-81eb-79936cbcca04'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:33.541816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] The token cache is being invalidate because project f223017dc9f7489a8089463f44fd53a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:33.542530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:33.544538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0b2a328-97ef-48bd-81eb-79936cbcca04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:33.547826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 880/1765] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:33 2022] DELETE /identity/v3/projects/f223017dc9f7489a8089463f44fd53a9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:33.552627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d48794-a859-4106-a253-06c99aa5a1ee None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:33.553070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d48794-a859-4106-a253-06c99aa5a1ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:33.553407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d48794-a859-4106-a253-06c99aa5a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:33.578957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-41d48794-a859-4106-a253-06c99aa5a1ee None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:33.579955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-41d48794-a859-4106-a253-06c99aa5a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:33.662962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 886/1766] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:33.671987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03d478-7851-4be1-bf26-ef6bb2f189a7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:33.672425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03d478-7851-4be1-bf26-ef6bb2f189a7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:33.672849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03d478-7851-4be1-bf26-ef6bb2f189a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:33.700312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd03d478-7851-4be1-bf26-ef6bb2f189a7 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:33.702907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd03d478-7851-4be1-bf26-ef6bb2f189a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:33.718299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d48794-a859-4106-a253-06c99aa5a1ee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:33.718438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 881/1767] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:33.739373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d48794-a859-4106-a253-06c99aa5a1ee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:33.753223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:33.755188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:33.755460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:33.755697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:33.757184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:33.757752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:33.759836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:33.773288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 887/1768] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:56:33.781636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03d478-7851-4be1-bf26-ef6bb2f189a7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:33.803259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03d478-7851-4be1-bf26-ef6bb2f189a7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:33.809185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:33.810103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:33.810541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:33.810970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:33.812464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:33.815021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:33.854795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:33.866546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 882/1769] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:33 2022] POST /identity/v3/projects => generated 363 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:33.867433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c27ff-3f1a-44eb-8bda-82d7c4bea42a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:33.959843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:33.960541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:33.960911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:33.961269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:33.962079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:33.964837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.000918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 888/1770] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:33 2022] POST /identity/v3/users => generated 282 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:34.005967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.028838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c07c0-f813-4abb-b406-fea2deb49b6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:34.038150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:34.039287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.039634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.039984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.041141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:34.042708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.046436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.050058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:34.060002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 883/1771] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:34.064280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.090509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:34.113884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4be1a4-d7e2-4242-a392-915dccc7f408 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:34.123083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6636-63da-4ecb-87c6-4e4f3614c720 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:34.130221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:34.131099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.135386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.137417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] PATH_INFO: `/v3/projects/a387c957936242528b0b3a90a1059586` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.141004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a387c957936242528b0b3a90a1059586)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:34.149406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:34.149816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.150105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.150446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.151188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:34.151670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.155371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.187979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 884/1772] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:56:34.190676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.228234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29afbab-6a91-47d4-a5bf-e0c3b4488c3e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:34.235751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66cc21-6125-49aa-a864-9209538e5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:34.236707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66cc21-6125-49aa-a864-9209538e5fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.236998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66cc21-6125-49aa-a864-9209538e5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.237322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66cc21-6125-49aa-a864-9209538e5fa6 admin admin] PATH_INFO: `/v3/projects/37ce9982a9a24b6dbee6407da433b2d1/users/e81be9a1e3334542beda9c1559a95d24/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.257883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66cc21-6125-49aa-a864-9209538e5fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ce9982a9a24b6dbee6407da433b2d1, user_id=e81be9a1e3334542beda9c1559a95d24, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:34.260920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66cc21-6125-49aa-a864-9209538e5fa6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.288021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 885/1773] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:34 2022] PUT /identity/v3/projects/37ce9982a9a24b6dbee6407da433b2d1/users/e81be9a1e3334542beda9c1559a95d24/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:34.300258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.300561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.300828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.308556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a387c957936242528b0b3a90a1059586', 'request_id': 'req-020f3ac0-cae9-49ff-af89-aaaed070b4b7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.329387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a387c957936242528b0b3a90a1059586', 'request_id': 'req-020f3ac0-cae9-49ff-af89-aaaed070b4b7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.341506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a387c957936242528b0b3a90a1059586', 'request_id': 'req-020f3ac0-cae9-49ff-af89-aaaed070b4b7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.368370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] The token cache is being invalidate because project a387c957936242528b0b3a90a1059586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:34.368677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.369471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.375641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 889/1774] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:34 2022] DELETE /identity/v3/projects/a387c957936242528b0b3a90a1059586 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:34.376431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.385710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:34.386876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] MFA Rules not processed for user `e81be9a1e3334542beda9c1559a95d24`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:34.389451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:34.429007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:34.533371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f3ac0-cae9-49ff-af89-aaaed070b4b7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:34.534733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 886/1775] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:34.535822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.537489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:34.641037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:34.642068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.642457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.642841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] PATH_INFO: `/v3/users/c766af119e1146d9ac3860d880f699e2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.644251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c766af119e1146d9ac3860d880f699e2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:34.646545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.674894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c6822-1948-4f42-b5bc-ce5ed32fdad5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:34.686837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:34.688033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.688033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.688264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.689378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:34.691360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.706882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:34.724754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c766af119e1146d9ac3860d880f699e2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.741525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 887/1776] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:34 2022] POST /identity/v3/projects => generated 363 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:34.747924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.770096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c766af119e1146d9ac3860d880f699e2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.771983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c766af119e1146d9ac3860d880f699e2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.773360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c766af119e1146d9ac3860d880f699e2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.793812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c766af119e1146d9ac3860d880f699e2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:34.822605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 890/1777] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:34 2022] DELETE /identity/v3/users/c766af119e1146d9ac3860d880f699e2 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:34.829805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb3ee5-9a40-4163-8a11-78bb748657ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:34.937767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:34.938584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:34.939102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:34.939477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:34.940488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:34.943146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:34.952297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:34.985534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:34.988206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:34.996993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 888/1778] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:34 2022] POST /identity/v3/users => generated 288 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:35.013485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46535bd0-a2d9-4ba8-96ae-ca8791282212 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:35.023892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:35.024777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.025292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.025688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.026962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:35.034002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:35.065578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 891/1779] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:35.069897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:35.080903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9b31d-b81c-4b63-b93d-dcac7ca24c30 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:35.086439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:35.087515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.088010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.088381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] PATH_INFO: `/v3/projects/c1a8c3ae41a349d4beadf3c3563a1bcb/users/8d226e4fe3e64f768f3a98de6b2ad5eb/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.106047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a8c3ae41a349d4beadf3c3563a1bcb, user_id=8d226e4fe3e64f768f3a98de6b2ad5eb, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:35.109326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:35.129511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 889/1780] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:35 2022] PUT /identity/v3/projects/c1a8c3ae41a349d4beadf3c3563a1bcb/users/8d226e4fe3e64f768f3a98de6b2ad5eb/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:35.134047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:35.158281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72498f-3a9f-4534-a4e2-23c4e7841a73 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:35.331187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85730b-7a7e-4f90-8d1e-5e1c2848ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:35.331876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85730b-7a7e-4f90-8d1e-5e1c2848ed9f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.332164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85730b-7a7e-4f90-8d1e-5e1c2848ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.332444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85730b-7a7e-4f90-8d1e-5e1c2848ed9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.333202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85730b-7a7e-4f90-8d1e-5e1c2848ed9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:35.334832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85730b-7a7e-4f90-8d1e-5e1c2848ed9f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:35.359126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 892/1781] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:35.363470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:35.380306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a0359-0c91-4aa9-9056-bf0a18a8b01b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:35.394346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:35.395946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.397587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.398043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] PATH_INFO: `/v3/projects/c1a8c3ae41a349d4beadf3c3563a1bcb/users/8d226e4fe3e64f768f3a98de6b2ad5eb/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.410002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a8c3ae41a349d4beadf3c3563a1bcb, user_id=8d226e4fe3e64f768f3a98de6b2ad5eb, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:35.416846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:35.437282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 890/1782] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:35 2022] PUT /identity/v3/projects/c1a8c3ae41a349d4beadf3c3563a1bcb/users/8d226e4fe3e64f768f3a98de6b2ad5eb/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:35.449011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac0c8-cec2-4236-a792-f46f6b3432ae None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.449625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac0c8-cec2-4236-a792-f46f6b3432ae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.450085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac0c8-cec2-4236-a792-f46f6b3432ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.500485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2cdac0c8-cec2-4236-a792-f46f6b3432ae None None] MFA Rules not processed for user `8d226e4fe3e64f768f3a98de6b2ad5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:35.503545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2cdac0c8-cec2-4236-a792-f46f6b3432ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:35.601057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 893/1783] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:35 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:35.613259 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:35.615168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:35.692423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ac67-2e93-4c5f-b3ca-8d21113e6b80 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:35.696257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:35.697188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.697543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.697890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.709827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:35.711482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:35.724572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 891/1784] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:35 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:56:35.814300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac0c8-cec2-4236-a792-f46f6b3432ae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:35.824319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac0c8-cec2-4236-a792-f46f6b3432ae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:35.828218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317b646-d299-4d15-a7d6-b71bd532054b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:35.829002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b646-d299-4d15-a7d6-b71bd532054b service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.830336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b646-d299-4d15-a7d6-b71bd532054b service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.830336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b646-d299-4d15-a7d6-b71bd532054b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.831935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b317b646-d299-4d15-a7d6-b71bd532054b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:35.862419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:35.945063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317b646-d299-4d15-a7d6-b71bd532054b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:35.948494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317b646-d299-4d15-a7d6-b71bd532054b service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:35.973243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 894/1785] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:35 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:36.370151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:36.372061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:36.404648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:36.406308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:36.407629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:36.490245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] MFA Rules not processed for user `3b0ba1e92d8e4663b8bb77bc90e80cda`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:36.494352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:36.547816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dabe70-5b43-4b29-94f8-c1f66a13af1c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:36.560313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:36.562029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:36.563131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:36.570716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] PATH_INFO: `/v3/projects/7c9b3fc9d01a41f992304fa82153c9e0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:36.572174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c9b3fc9d01a41f992304fa82153c9e0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:36.574334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:36.665023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c9b3fc9d01a41f992304fa82153c9e0', 'request_id': 'req-3bc45495-2162-4613-9a89-a00ab165ef73'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:36.670962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 895/1786] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:36.672063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:36.674472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:36.708803 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c9b3fc9d01a41f992304fa82153c9e0', 'request_id': 'req-3bc45495-2162-4613-9a89-a00ab165ef73'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:36.710653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=6,keystone:SELECT=89,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:36.717510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6f950-dfd0-4e25-a591-5e7a34b0f578 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:36.719251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c9b3fc9d01a41f992304fa82153c9e0', 'request_id': 'req-3bc45495-2162-4613-9a89-a00ab165ef73'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:36.740644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] The token cache is being invalidate because project 7c9b3fc9d01a41f992304fa82153c9e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:36.741659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:36.743820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc45495-2162-4613-9a89-a00ab165ef73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:36.753900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 892/1787] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:36 2022] DELETE /identity/v3/projects/7c9b3fc9d01a41f992304fa82153c9e0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:36.757844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d4837-3bd0-4082-8f16-e7fbadff1ed0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:36.758212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d4837-3bd0-4082-8f16-e7fbadff1ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:36.758564 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d4837-3bd0-4082-8f16-e7fbadff1ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:36.810819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b7d4837-3bd0-4082-8f16-e7fbadff1ed0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:36.815170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b7d4837-3bd0-4082-8f16-e7fbadff1ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:36.882085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:36.883323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:36.885509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:36.885881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:36.887113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:36.892247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:36.965390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 896/1788] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:36 2022] POST /identity/v3/projects => generated 403 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:36.967173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:36.973546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:37.034828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 893/1789] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:37.047706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24710f-a093-47fe-a72c-f3426573ea51 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.048006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24710f-a093-47fe-a72c-f3426573ea51 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.048327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24710f-a093-47fe-a72c-f3426573ea51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.078715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:37.087751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dc24710f-a093-47fe-a72c-f3426573ea51 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:37.093721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc24710f-a093-47fe-a72c-f3426573ea51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:37.161657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae696a2-e10e-414b-a641-40ff9792530b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.179203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.181575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.184378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.184827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.185812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.190450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.268764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 897/1790] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:36 2022] POST /identity/v3/users => generated 308 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:37.272301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.286545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 894/1791] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:37.287541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24710f-a093-47fe-a72c-f3426573ea51 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.303478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96334c-c9fe-4452-bfa1-960cd349ee64 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.317618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24710f-a093-47fe-a72c-f3426573ea51 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.322039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.322738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.323026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.323303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.324047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.333950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.336048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.336732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.337015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.337321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.338063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:37.338372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.339998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.361248 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 898/1792] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:37.364614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.383445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 895/1793] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:37.384631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.403175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cfd2b-9931-44fc-9206-cb631cce40de admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.407531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.408451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.408739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.409020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] PATH_INFO: `/v3/projects/85d1284536e14774843dd9c75ba05998/users/8f321b8e1c924d4a93789106bdfa9d0d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.421460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6ea86-be73-4c06-b5e1-634b2ce067a1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.438204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.439197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.439488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.439699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d1284536e14774843dd9c75ba05998, user_id=8f321b8e1c924d4a93789106bdfa9d0d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.439800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.440452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.442372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.442492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.475602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 899/1794] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:37 2022] PUT /identity/v3/projects/85d1284536e14774843dd9c75ba05998/users/8f321b8e1c924d4a93789106bdfa9d0d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:37.478172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.518674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 896/1795] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:37 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:37.522689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.618401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf2201-a9b1-48de-af16-247a91bea4c3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.721698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:37.752569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.753496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.754112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.754535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.755461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.761729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.786372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5918c-485c-44ad-b711-a0ae9b15f63f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.790346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 900/1796] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:56:37.794496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.804917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983a2aa-5954-4e9e-b9ac-7df9cf2378d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.807135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a2aa-5954-4e9e-b9ac-7df9cf2378d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.808568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a2aa-5954-4e9e-b9ac-7df9cf2378d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.809010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a2aa-5954-4e9e-b9ac-7df9cf2378d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.811663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca8db3-8869-43fd-b086-e41032266805 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.812398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983a2aa-5954-4e9e-b9ac-7df9cf2378d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.825911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983a2aa-5954-4e9e-b9ac-7df9cf2378d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.827495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.828446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.828805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.829094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] PATH_INFO: `/v3/projects/85d1284536e14774843dd9c75ba05998/users/8f321b8e1c924d4a93789106bdfa9d0d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.842070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d1284536e14774843dd9c75ba05998, user_id=8f321b8e1c924d4a93789106bdfa9d0d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.846873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.866707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 901/1797] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:37 2022] PUT /identity/v3/projects/85d1284536e14774843dd9c75ba05998/users/8f321b8e1c924d4a93789106bdfa9d0d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:37.877258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 897/1798] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:37 2022] POST /identity/v3/users => generated 288 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:37.882389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.919658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353059f-c660-4f6c-9476-21bc93453885 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.920028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353059f-c660-4f6c-9476-21bc93453885 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.920459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353059f-c660-4f6c-9476-21bc93453885 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.924007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 898/1799] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:56:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:56:37.926819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:38.011117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82804b22-907c-449e-97df-ffc9e56fe4ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:38.013012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82804b22-907c-449e-97df-ffc9e56fe4ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:38.013352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82804b22-907c-449e-97df-ffc9e56fe4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:38.054295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-82804b22-907c-449e-97df-ffc9e56fe4ab None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:38.055433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c40af8-f24a-4670-9df4-f1d3f78c52b0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:38.055547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-82804b22-907c-449e-97df-ffc9e56fe4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:38.069685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:38.070571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:38.070865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:38.071149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:38.072281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:38.075087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:38.104916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 902/1800] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:38.109561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:38.134577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146967-b434-45e6-b226-8315de80499e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:38.155001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:38.158936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:38.159671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:38.159790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:38.163082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:38.166341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:38.223632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 903/1801] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:38.227853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:38.246865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 899/1802] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:56:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:38.255297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cad1ea-1902-4ba3-a69f-a9bfbb5d6c81 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:38.261067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:38.262604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:38.262922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:38.263345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] PATH_INFO: `/v3/projects/0926e6ea11a2454e9a5b4ef6c5fd2d96/users/f90654dfe0c94d2ab61872dd244f4d81/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:38.284850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0926e6ea11a2454e9a5b4ef6c5fd2d96, user_id=f90654dfe0c94d2ab61872dd244f4d81, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:38.288344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:38.313238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 904/1803] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:38 2022] PUT /identity/v3/projects/0926e6ea11a2454e9a5b4ef6c5fd2d96/users/f90654dfe0c94d2ab61872dd244f4d81/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:38.320759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:38.321081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:38.321382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:38.358296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] MFA Rules not processed for user `f90654dfe0c94d2ab61872dd244f4d81`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:38.359893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:38.481987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 900/1804] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:38.493295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:38.497791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:38.615409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d591a7-6a57-4e3e-afc6-be29b6eab248 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:38.626991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe50f8f-2990-4c2e-a19f-e5f98702c26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:38.629431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe50f8f-2990-4c2e-a19f-e5f98702c26e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:38.629980 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe50f8f-2990-4c2e-a19f-e5f98702c26e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:38.630352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe50f8f-2990-4c2e-a19f-e5f98702c26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:38.656726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe50f8f-2990-4c2e-a19f-e5f98702c26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:38.658452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe50f8f-2990-4c2e-a19f-e5f98702c26e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:38.681208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 905/1805] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:38.966002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:38.969417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:39.132326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:39.186011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1e976-e1f9-401e-a600-5b3613ed6df1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:39.220934 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138e746-b1c6-43d1-9d8c-13b5e1424c22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:39.221850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138e746-b1c6-43d1-9d8c-13b5e1424c22 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:39.222620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138e746-b1c6-43d1-9d8c-13b5e1424c22 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:39.222620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138e746-b1c6-43d1-9d8c-13b5e1424c22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:39.237935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138e746-b1c6-43d1-9d8c-13b5e1424c22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:39.239698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138e746-b1c6-43d1-9d8c-13b5e1424c22 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:39.254293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 901/1806] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:40.290907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa2d11-db96-4b9a-a39a-afaa54fa00ec None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:40.291399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa2d11-db96-4b9a-a39a-afaa54fa00ec None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:40.291508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa2d11-db96-4b9a-a39a-afaa54fa00ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:40.316843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-91fa2d11-db96-4b9a-a39a-afaa54fa00ec None None] MFA Rules not processed for user `f90654dfe0c94d2ab61872dd244f4d81`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:40.318121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-91fa2d11-db96-4b9a-a39a-afaa54fa00ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:40.325889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 906/1807] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:40.470525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf594f1-2066-40fc-b9b7-80900da1c08e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:40.471390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf594f1-2066-40fc-b9b7-80900da1c08e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:40.471676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf594f1-2066-40fc-b9b7-80900da1c08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:40.509009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5cf594f1-2066-40fc-b9b7-80900da1c08e None None] MFA Rules not processed for user `8f321b8e1c924d4a93789106bdfa9d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:40.510340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5cf594f1-2066-40fc-b9b7-80900da1c08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:40.596721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 902/1808] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:41.504157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e0cbc-17f4-4ba1-bd2c-4f22b88b169e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:41.505020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e0cbc-17f4-4ba1-bd2c-4f22b88b169e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:41.505328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e0cbc-17f4-4ba1-bd2c-4f22b88b169e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:41.507162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 907/1809] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:56:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:41.538084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246bfa-28ca-4a44-9789-fb1f1b77a468 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:41.538644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246bfa-28ca-4a44-9789-fb1f1b77a468 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:41.538644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246bfa-28ca-4a44-9789-fb1f1b77a468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:41.561381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-32246bfa-28ca-4a44-9789-fb1f1b77a468 None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:41.562765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32246bfa-28ca-4a44-9789-fb1f1b77a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:41.639492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 903/1810] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:56:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:41.650586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e0cbc-17f4-4ba1-bd2c-4f22b88b169e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:41.652381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b16e0cbc-17f4-4ba1-bd2c-4f22b88b169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:41.675488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e0cbc-17f4-4ba1-bd2c-4f22b88b169e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:41.681579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:41.682430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:41.682859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:41.683383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:41.694639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:41.696440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:41.706727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 908/1811] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:42.089014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246bfa-28ca-4a44-9789-fb1f1b77a468 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:42.203277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246bfa-28ca-4a44-9789-fb1f1b77a468 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:42.208324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:42.209439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:42.209725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:42.210126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] PATH_INFO: `/v3/users/3b0ba1e92d8e4663b8bb77bc90e80cda` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:42.211336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b0ba1e92d8e4663b8bb77bc90e80cda)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:42.213317 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:42.225543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:42.279368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b0ba1e92d8e4663b8bb77bc90e80cda'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.294756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b0ba1e92d8e4663b8bb77bc90e80cda'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.321176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b0ba1e92d8e4663b8bb77bc90e80cda'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.322046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b0ba1e92d8e4663b8bb77bc90e80cda'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.322769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b0ba1e92d8e4663b8bb77bc90e80cda'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.339302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 904/1812] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:42 2022] DELETE /identity/v3/users/3b0ba1e92d8e4663b8bb77bc90e80cda => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:42.540055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:42.551600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52b73-9c03-457a-9a4b-99999d3ba912 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:42.607441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:42.608859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:42.609171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:42.609455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] PATH_INFO: `/v3/projects/55c936bd8e71491da96a2abe726918a2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:42.610647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c936bd8e71491da96a2abe726918a2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:42.612566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:42.618732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:42.659269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c936bd8e71491da96a2abe726918a2', 'request_id': 'req-4334889e-dd73-4a2a-bc39-7325f4c738b5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.665735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c936bd8e71491da96a2abe726918a2', 'request_id': 'req-4334889e-dd73-4a2a-bc39-7325f4c738b5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.671858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c936bd8e71491da96a2abe726918a2', 'request_id': 'req-4334889e-dd73-4a2a-bc39-7325f4c738b5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.687115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] The token cache is being invalidate because project 55c936bd8e71491da96a2abe726918a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:42.687401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.688035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.689669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 909/1813] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:42 2022] DELETE /identity/v3/projects/55c936bd8e71491da96a2abe726918a2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:42.693626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:42.695047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:42.710267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dc473-a223-45df-809a-719b29fe6fb7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:42.786041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:42.786930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:42.787216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:42.787493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] PATH_INFO: `/v3/users/8f321b8e1c924d4a93789106bdfa9d0d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:42.788700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f321b8e1c924d4a93789106bdfa9d0d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:42.790734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:42.840741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f321b8e1c924d4a93789106bdfa9d0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.847771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f321b8e1c924d4a93789106bdfa9d0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.861475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f321b8e1c924d4a93789106bdfa9d0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.862293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f321b8e1c924d4a93789106bdfa9d0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.863072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f28e35a-3261-4a18-9edc-6ebd62cef1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f321b8e1c924d4a93789106bdfa9d0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:42.876608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 905/1814] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:42 2022] DELETE /identity/v3/users/8f321b8e1c924d4a93789106bdfa9d0d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:43.043738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.045245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:43.060526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334889e-dd73-4a2a-bc39-7325f4c738b5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.119590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.120459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.120743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.121025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] PATH_INFO: `/v3/projects/85d1284536e14774843dd9c75ba05998` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.122267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d1284536e14774843dd9c75ba05998)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.124203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.175639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d1284536e14774843dd9c75ba05998', 'request_id': 'req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:43.183682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d1284536e14774843dd9c75ba05998', 'request_id': 'req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:43.189665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d1284536e14774843dd9c75ba05998', 'request_id': 'req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:43.209449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] The token cache is being invalidate because project 85d1284536e14774843dd9c75ba05998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:43.209881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:43.210867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0d5303d7-4e0e-44b2-ad51-8c0100d58c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:43.212637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 910/1815] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:43 2022] DELETE /identity/v3/projects/85d1284536e14774843dd9c75ba05998 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:43.219570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a066-c769-4519-bf69-8a2fdf73ac6e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.219815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a066-c769-4519-bf69-8a2fdf73ac6e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.220093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a066-c769-4519-bf69-8a2fdf73ac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.239643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:43.242480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c907a066-c769-4519-bf69-8a2fdf73ac6e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:43.246604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c907a066-c769-4519-bf69-8a2fdf73ac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:43.311462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 906/1816] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:43.320302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef86ff-bbb7-430a-baa9-c51fb1015cea None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.320594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef86ff-bbb7-430a-baa9-c51fb1015cea None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.320877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef86ff-bbb7-430a-baa9-c51fb1015cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.339877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8bef86ff-bbb7-430a-baa9-c51fb1015cea None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:43.341095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8bef86ff-bbb7-430a-baa9-c51fb1015cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:43.353031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 911/1817] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:43.357683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a066-c769-4519-bf69-8a2fdf73ac6e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.367528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a066-c769-4519-bf69-8a2fdf73ac6e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.372416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.373080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.374752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.374752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.379300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:43.379300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.379636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.393235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 907/1818] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:43.397465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef86ff-bbb7-430a-baa9-c51fb1015cea None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.407933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef86ff-bbb7-430a-baa9-c51fb1015cea None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.413069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.413842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.414196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.414480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.415216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.417233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.437921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:43.458057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 912/1819] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:43 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:43.460456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.470211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808252b1-0967-4b63-9abc-179221766ddb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.532067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.532740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.533020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.533401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.534188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.536076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.568302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 908/1820] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:43 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:43.572780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.585575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d23bb3-e6b8-494b-9697-c557f59be605 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.592141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.593581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.593947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.594283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.595086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.596776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.607634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 913/1821] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:43.611837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.624592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67351f2-19ac-432e-b970-5afb563812c7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.630900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73047-a1bf-4e89-aeee-9808d5907239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.631628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73047-a1bf-4e89-aeee-9808d5907239 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.631948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73047-a1bf-4e89-aeee-9808d5907239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.632272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73047-a1bf-4e89-aeee-9808d5907239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.632913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73047-a1bf-4e89-aeee-9808d5907239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.634563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73047-a1bf-4e89-aeee-9808d5907239 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.657867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 909/1822] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:43.659395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.670461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2378f7-e3f5-4865-ac23-afea4e07d2d3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.676428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.677446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.677791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.678134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] PATH_INFO: `/v3/projects/aa80592f90f54012ae709e6dad225ea8/users/13889d6af7e446d29b35c77edc4c7ac3/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.694458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa80592f90f54012ae709e6dad225ea8, user_id=13889d6af7e446d29b35c77edc4c7ac3, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.697109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.710370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 914/1823] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:43 2022] PUT /identity/v3/projects/aa80592f90f54012ae709e6dad225ea8/users/13889d6af7e446d29b35c77edc4c7ac3/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:43.722108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308e56-0b9f-4a33-b010-f5f090de36dc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.722626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308e56-0b9f-4a33-b010-f5f090de36dc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.722715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308e56-0b9f-4a33-b010-f5f090de36dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.761350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-30308e56-0b9f-4a33-b010-f5f090de36dc None None] MFA Rules not processed for user `13889d6af7e446d29b35c77edc4c7ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:43.761350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-30308e56-0b9f-4a33-b010-f5f090de36dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:43.834031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 910/1824] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:43.838615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.900240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381d71-18a6-47c2-9ce9-c6310ca36f4b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:43.905725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:43.906387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:43.906679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:43.906959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:43.907644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:43.909591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:43.940860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 915/1825] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:43 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:43.945135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30308e56-0b9f-4a33-b010-f5f090de36dc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:43.952201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:43.958949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30308e56-0b9f-4a33-b010-f5f090de36dc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:44.022680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:44.023306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.023616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.023917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.024692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:44.026947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:44.049663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 911/1826] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:43 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:44.054091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:44.064627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7357df-5243-46f7-b3ff-b96058c6a41d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:44.069750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:44.070465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.070751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.071029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.071759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:44.073386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:44.086166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 916/1827] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:56:44.090102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:44.100623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995027-d961-403f-81f3-83a317868b6a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:44.105210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:44.106279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.106562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.106855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] PATH_INFO: `/v3/projects/87419fd06eee43808f949e3f3fc69523/users/bb262e58aeb741fa948f5d6bfc1be94a/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.122063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87419fd06eee43808f949e3f3fc69523, user_id=bb262e58aeb741fa948f5d6bfc1be94a, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:44.124885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:44.142292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 912/1828] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:44 2022] PUT /identity/v3/projects/87419fd06eee43808f949e3f3fc69523/users/bb262e58aeb741fa948f5d6bfc1be94a/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:44.145942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:44.156795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbd212-a073-4ff1-98ce-34c30ce34ec3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:44.220358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdcab4-2e07-45d0-b96d-2a9809b3848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:44.221038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdcab4-2e07-45d0-b96d-2a9809b3848d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.221372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdcab4-2e07-45d0-b96d-2a9809b3848d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.221647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdcab4-2e07-45d0-b96d-2a9809b3848d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.222362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdcab4-2e07-45d0-b96d-2a9809b3848d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:44.223968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdcab4-2e07-45d0-b96d-2a9809b3848d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:44.233499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 917/1829] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:56:44.237744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:44.247891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e0c94-4ad2-45e2-a49d-ab0155520b33 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:44.252472 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:44.253409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.253824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.253943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] PATH_INFO: `/v3/projects/87419fd06eee43808f949e3f3fc69523/users/bb262e58aeb741fa948f5d6bfc1be94a/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.255886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=87419fd06eee43808f949e3f3fc69523, user_id=bb262e58aeb741fa948f5d6bfc1be94a, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:44.258591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:44.270149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 913/1830] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:44 2022] PUT /identity/v3/projects/87419fd06eee43808f949e3f3fc69523/users/bb262e58aeb741fa948f5d6bfc1be94a/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:44.282700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d14f5e-cbe2-449b-be9a-8bcd1c9f674a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.283119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d14f5e-cbe2-449b-be9a-8bcd1c9f674a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.283477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d14f5e-cbe2-449b-be9a-8bcd1c9f674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.315490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-33d14f5e-cbe2-449b-be9a-8bcd1c9f674a None None] MFA Rules not processed for user `bb262e58aeb741fa948f5d6bfc1be94a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:44.316736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33d14f5e-cbe2-449b-be9a-8bcd1c9f674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:44.339370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:44.387277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 918/1831] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:44.399121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:44.400874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:44.458250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62260b-b98f-4661-ace8-a635e4bcc9be admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:44.461891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6974118-4481-465f-8113-eba92731577e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:44.462541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6974118-4481-465f-8113-eba92731577e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.462828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6974118-4481-465f-8113-eba92731577e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.463105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6974118-4481-465f-8113-eba92731577e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.473288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6974118-4481-465f-8113-eba92731577e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:44.474739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6974118-4481-465f-8113-eba92731577e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:44.486116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 914/1832] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:44.901425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d14f5e-cbe2-449b-be9a-8bcd1c9f674a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:44.962148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d14f5e-cbe2-449b-be9a-8bcd1c9f674a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:44.966597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:44.967470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:44.967750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:44.968023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] PATH_INFO: `/v3/users/13889d6af7e446d29b35c77edc4c7ac3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:44.969271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13889d6af7e446d29b35c77edc4c7ac3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:44.971229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:45.017141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13889d6af7e446d29b35c77edc4c7ac3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.030319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13889d6af7e446d29b35c77edc4c7ac3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.031028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13889d6af7e446d29b35c77edc4c7ac3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.031639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13889d6af7e446d29b35c77edc4c7ac3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.039876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13889d6af7e446d29b35c77edc4c7ac3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.052517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 919/1833] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:44 2022] DELETE /identity/v3/users/13889d6af7e446d29b35c77edc4c7ac3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:45.057044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:45.057555 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:45.057758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:45.076185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:45.077429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:45.127089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:45.137516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 915/1834] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:45.147992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:45.149466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:45.204218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:45.230948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21405-a761-4f77-bc3a-16f1b638a727 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:45.235247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:45.235898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:45.236178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:45.236476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:45.246206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:45.247658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:45.257644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 920/1835] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:45.569023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:45.571102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:45.588530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42596b52-48d8-4d49-895e-24fb25b5f277 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:45.594546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:45.595502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:45.595693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:45.595969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] PATH_INFO: `/v3/projects/aa80592f90f54012ae709e6dad225ea8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:45.597164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa80592f90f54012ae709e6dad225ea8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:45.599114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:45.645640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa80592f90f54012ae709e6dad225ea8', 'request_id': 'req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.653720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa80592f90f54012ae709e6dad225ea8', 'request_id': 'req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.660560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa80592f90f54012ae709e6dad225ea8', 'request_id': 'req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.670976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] The token cache is being invalidate because project aa80592f90f54012ae709e6dad225ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:45.671293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.671924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.673499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 916/1836] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:45 2022] DELETE /identity/v3/projects/aa80592f90f54012ae709e6dad225ea8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:45.677490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:45.687274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6af3-cdf4-485a-bbe9-fafc827f5999 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:45.746693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:45.747581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:45.747872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:45.748152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] PATH_INFO: `/v3/users/bb262e58aeb741fa948f5d6bfc1be94a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:45.749385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb262e58aeb741fa948f5d6bfc1be94a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:45.751325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:45.799323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb262e58aeb741fa948f5d6bfc1be94a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.812784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb262e58aeb741fa948f5d6bfc1be94a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.813463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb262e58aeb741fa948f5d6bfc1be94a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.814082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb262e58aeb741fa948f5d6bfc1be94a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.821553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e44d8c91-8ae6-4c10-b443-17339c01b08e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb262e58aeb741fa948f5d6bfc1be94a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:45.837717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 921/1837] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:45 2022] DELETE /identity/v3/users/bb262e58aeb741fa948f5d6bfc1be94a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:46.194167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.195699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:46.209970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f5d3d-0a02-4ea7-9379-0401d55fc10c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.267389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.268279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.268559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.268829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] PATH_INFO: `/v3/projects/87419fd06eee43808f949e3f3fc69523` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.270035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87419fd06eee43808f949e3f3fc69523)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.271993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:46.317084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87419fd06eee43808f949e3f3fc69523', 'request_id': 'req-cb638859-437d-4fcc-b3bf-e8d63286fbc4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:46.323403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87419fd06eee43808f949e3f3fc69523', 'request_id': 'req-cb638859-437d-4fcc-b3bf-e8d63286fbc4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:46.329485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87419fd06eee43808f949e3f3fc69523', 'request_id': 'req-cb638859-437d-4fcc-b3bf-e8d63286fbc4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:46.339435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] The token cache is being invalidate because project 87419fd06eee43808f949e3f3fc69523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:46.339733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:46.340332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb638859-437d-4fcc-b3bf-e8d63286fbc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:46.341884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 917/1838] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:46 2022] DELETE /identity/v3/projects/87419fd06eee43808f949e3f3fc69523 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:46.346716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45858-4e7c-4ea1-801b-91086dde22f0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.347175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45858-4e7c-4ea1-801b-91086dde22f0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.347281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45858-4e7c-4ea1-801b-91086dde22f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.366306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e5f45858-4e7c-4ea1-801b-91086dde22f0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:46.367530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e5f45858-4e7c-4ea1-801b-91086dde22f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:46.417857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:46.430862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 922/1839] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:46.439775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95cac-895f-4d61-904f-8da24ba86776 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.440065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95cac-895f-4d61-904f-8da24ba86776 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.440344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95cac-895f-4d61-904f-8da24ba86776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.461058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b3a95cac-895f-4d61-904f-8da24ba86776 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:46.462574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3a95cac-895f-4d61-904f-8da24ba86776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:46.480272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 918/1840] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:46.480617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45858-4e7c-4ea1-801b-91086dde22f0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.491930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45858-4e7c-4ea1-801b-91086dde22f0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.496805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.499300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.499300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.499602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.500133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:46.500936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.502118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:46.518574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 923/1841] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:56:46.519191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95cac-895f-4d61-904f-8da24ba86776 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.524884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:46.529687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95cac-895f-4d61-904f-8da24ba86776 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.536994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.538043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.538578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.538860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.539605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.541647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:46.579054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 919/1842] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:46 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:46.580317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.590153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec12d0-8da5-4284-9029-f1571862ef1c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.656324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.657044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.657362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.657714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.658486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.660410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:46.682198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 924/1843] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:46 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:46.686478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.696877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c2ac-246d-448b-b63f-bea913259548 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.701572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.702366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.702650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.702920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.703638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.705263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:46.716778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 920/1844] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:46.721236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.731311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dcbe7-c182-41a4-a8e6-72b0cc673f85 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.736727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8fa00-e9c5-4dd4-b211-f75b90070fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.737424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8fa00-e9c5-4dd4-b211-f75b90070fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.737706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8fa00-e9c5-4dd4-b211-f75b90070fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.737973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8fa00-e9c5-4dd4-b211-f75b90070fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.738701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8fa00-e9c5-4dd4-b211-f75b90070fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.740283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8fa00-e9c5-4dd4-b211-f75b90070fb7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:46.752051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 925/1845] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:46.756179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.765957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b255144-d93a-4a3d-9fd9-adc93c71aa8e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.771371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.772262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.772546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.772823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] PATH_INFO: `/v3/projects/68517d0619fc4e7391e1952f7ff0dc79/users/78122592b73a4419b6d530ab61554503/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.788212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=68517d0619fc4e7391e1952f7ff0dc79, user_id=78122592b73a4419b6d530ab61554503, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.790892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:46.804036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 921/1846] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:46 2022] PUT /identity/v3/projects/68517d0619fc4e7391e1952f7ff0dc79/users/78122592b73a4419b6d530ab61554503/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:46.814281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363a4e5-400d-460f-abf5-193d3ddf5cb5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.814567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363a4e5-400d-460f-abf5-193d3ddf5cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.814838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363a4e5-400d-460f-abf5-193d3ddf5cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.840026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9363a4e5-400d-460f-abf5-193d3ddf5cb5 None None] MFA Rules not processed for user `78122592b73a4419b6d530ab61554503`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:46.841141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9363a4e5-400d-460f-abf5-193d3ddf5cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:46.915957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 926/1847] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:46.921242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:46.987109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312c61d-7a17-4531-82ed-c63a62dc8aea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:46.992067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:46.992750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:46.993037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:46.993359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:46.994071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:46.996394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.028051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 922/1848] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:46 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:47.032434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363a4e5-400d-460f-abf5-193d3ddf5cb5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:47.043559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363a4e5-400d-460f-abf5-193d3ddf5cb5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:47.100049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:47.110601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:47.111290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.111581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.111857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.112556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:47.114650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.138917 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 927/1849] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:47 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:47.140655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:47.150757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3041df-fabf-435b-b118-9e166f556688 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:47.156388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:47.156813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.157093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.157380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.158107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:47.159704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.174343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 923/1850] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:47.175922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:47.189139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560da59c-5136-429d-841b-190b2c5c565f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:47.199343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:47.200282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.200554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.200820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] PATH_INFO: `/v3/projects/9a93d2fe75fb4e43b9fd147fd5c8d1a0/users/b42a0528467a4f06b93279294afef234/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.216376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a93d2fe75fb4e43b9fd147fd5c8d1a0, user_id=b42a0528467a4f06b93279294afef234, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:47.219539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.235772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 928/1851] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:47 2022] PUT /identity/v3/projects/9a93d2fe75fb4e43b9fd147fd5c8d1a0/users/b42a0528467a4f06b93279294afef234/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:56:47.237066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:47.247776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09378ef-2600-4390-b403-edee9f0d874d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:47.310002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef0033-0901-4119-aceb-697a8fce481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:47.310677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef0033-0901-4119-aceb-697a8fce481b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.310961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef0033-0901-4119-aceb-697a8fce481b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.311240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef0033-0901-4119-aceb-697a8fce481b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.311960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef0033-0901-4119-aceb-697a8fce481b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:47.313595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef0033-0901-4119-aceb-697a8fce481b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.323247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 924/1852] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:56:47.328003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:47.337990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158c183-01f1-4092-8800-f0cce64c6b2c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:47.343238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:47.344110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.344388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.344654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] PATH_INFO: `/v3/projects/9a93d2fe75fb4e43b9fd147fd5c8d1a0/users/b42a0528467a4f06b93279294afef234/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.346625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a93d2fe75fb4e43b9fd147fd5c8d1a0, user_id=b42a0528467a4f06b93279294afef234, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:47.349266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.359360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 929/1853] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:47 2022] PUT /identity/v3/projects/9a93d2fe75fb4e43b9fd147fd5c8d1a0/users/b42a0528467a4f06b93279294afef234/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:47.370102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5619047-e4a9-4f21-8460-10f1548677c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.370396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5619047-e4a9-4f21-8460-10f1548677c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.370682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5619047-e4a9-4f21-8460-10f1548677c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.400158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d5619047-e4a9-4f21-8460-10f1548677c7 None None] MFA Rules not processed for user `b42a0528467a4f06b93279294afef234`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:47.401411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d5619047-e4a9-4f21-8460-10f1548677c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:47.407225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:47.472562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 925/1854] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:47.484213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:47.485911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:47.546422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94b5da-f820-4bda-bb66-3c649721f998 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:47.550180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec890b-3d1c-47b8-9bcf-cc7abed22e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:47.550831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec890b-3d1c-47b8-9bcf-cc7abed22e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.551115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec890b-3d1c-47b8-9bcf-cc7abed22e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.551396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec890b-3d1c-47b8-9bcf-cc7abed22e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.561561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec890b-3d1c-47b8-9bcf-cc7abed22e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:47.563071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec890b-3d1c-47b8-9bcf-cc7abed22e80 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.573094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 930/1855] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:47.646938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5619047-e4a9-4f21-8460-10f1548677c7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:47.656548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5619047-e4a9-4f21-8460-10f1548677c7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:47.662180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9a2f6-b014-4c81-8613-2cbdacfcdef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:47.662819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a2f6-b014-4c81-8613-2cbdacfcdef4 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:47.663067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a2f6-b014-4c81-8613-2cbdacfcdef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:47.663346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9a2f6-b014-4c81-8613-2cbdacfcdef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:47.726007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9a2f6-b014-4c81-8613-2cbdacfcdef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:47.727481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9a2f6-b014-4c81-8613-2cbdacfcdef4 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:47.736727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 926/1856] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:48.689363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f1161-c08e-4286-9bfe-1265327eadf3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:48.689894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f1161-c08e-4286-9bfe-1265327eadf3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:48.690114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f1161-c08e-4286-9bfe-1265327eadf3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:48.691651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 931/1857] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:56:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:56:48.698389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1735a6-f1b3-40e3-84e0-f28e68a6c36c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:48.698810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1735a6-f1b3-40e3-84e0-f28e68a6c36c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:48.698950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1735a6-f1b3-40e3-84e0-f28e68a6c36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:48.718684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8a1735a6-f1b3-40e3-84e0-f28e68a6c36c None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:48.719831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a1735a6-f1b3-40e3-84e0-f28e68a6c36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:48.788282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 927/1858] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:48.800507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f1161-c08e-4286-9bfe-1265327eadf3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:48.802209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c08f1161-c08e-4286-9bfe-1265327eadf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:48.818912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f1161-c08e-4286-9bfe-1265327eadf3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:48.824143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb699561-6b12-47b0-98dd-28ab5d5b2911 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:48.824809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699561-6b12-47b0-98dd-28ab5d5b2911 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:48.825087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699561-6b12-47b0-98dd-28ab5d5b2911 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:48.825386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699561-6b12-47b0-98dd-28ab5d5b2911 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:48.834280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb699561-6b12-47b0-98dd-28ab5d5b2911 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:48.835857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb699561-6b12-47b0-98dd-28ab5d5b2911 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:48.846388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 932/1859] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:51.565541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1735a6-f1b3-40e3-84e0-f28e68a6c36c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:51.605801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:51.650400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1735a6-f1b3-40e3-84e0-f28e68a6c36c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:51.656401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:51.657337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:51.657621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:51.657917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] PATH_INFO: `/v3/users/78122592b73a4419b6d530ab61554503` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:51.659246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78122592b73a4419b6d530ab61554503)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:51.661183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:51.714316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78122592b73a4419b6d530ab61554503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:51.722202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78122592b73a4419b6d530ab61554503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:51.735542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78122592b73a4419b6d530ab61554503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:51.736206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78122592b73a4419b6d530ab61554503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:51.736816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78122592b73a4419b6d530ab61554503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:51.749553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 928/1860] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:51 2022] DELETE /identity/v3/users/78122592b73a4419b6d530ab61554503 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:51.754102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:51.754583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:51.754672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:51.771761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:51.772888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:51.831890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 933/1861] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:51.842011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:51.843651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:51.900812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace49de-4573-427f-bf8b-c98b27ffcb86 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:51.905217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:51.905863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:51.906148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:51.906418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:51.915189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:51.916606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:51.926532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 929/1862] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:52.213214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:52.214702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:52.226828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902eb8e-5d24-4ddd-9a7b-cc37146b5dc6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:52.231475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:52.232357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:52.232645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:52.232982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] PATH_INFO: `/v3/projects/68517d0619fc4e7391e1952f7ff0dc79` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:52.234237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=68517d0619fc4e7391e1952f7ff0dc79)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:52.236172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:52.281874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:52.296480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68517d0619fc4e7391e1952f7ff0dc79', 'request_id': 'req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.302682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68517d0619fc4e7391e1952f7ff0dc79', 'request_id': 'req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.308157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68517d0619fc4e7391e1952f7ff0dc79', 'request_id': 'req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.320515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] The token cache is being invalidate because project 68517d0619fc4e7391e1952f7ff0dc79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:52.320802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.321442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.322975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 934/1863] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:52 2022] DELETE /identity/v3/projects/68517d0619fc4e7391e1952f7ff0dc79 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:52.326948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:52.336404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29c06-b157-4298-a806-440c82df361c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:52.387274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:52.394881 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:52.396000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:52.396442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:52.396725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] PATH_INFO: `/v3/users/b42a0528467a4f06b93279294afef234` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:52.398126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42a0528467a4f06b93279294afef234)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:52.400035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:52.446641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42a0528467a4f06b93279294afef234'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.454655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42a0528467a4f06b93279294afef234'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.467513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42a0528467a4f06b93279294afef234'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.468186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42a0528467a4f06b93279294afef234'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.468829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aabc8178-f0b0-491b-83af-fa64b237f72d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42a0528467a4f06b93279294afef234'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:52.481442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 930/1864] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:52 2022] DELETE /identity/v3/users/b42a0528467a4f06b93279294afef234 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:52.913878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:52.915854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:52.942075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee411e-da1a-485b-b8c6-cf31b957ad9f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.025336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.026278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.026561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.026834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] PATH_INFO: `/v3/projects/9a93d2fe75fb4e43b9fd147fd5c8d1a0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.028285 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a93d2fe75fb4e43b9fd147fd5c8d1a0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.030764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.084995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a93d2fe75fb4e43b9fd147fd5c8d1a0', 'request_id': 'req-04b63059-8405-4fa0-82d0-c24cc1d8d399'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:53.091035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a93d2fe75fb4e43b9fd147fd5c8d1a0', 'request_id': 'req-04b63059-8405-4fa0-82d0-c24cc1d8d399'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:53.096952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a93d2fe75fb4e43b9fd147fd5c8d1a0', 'request_id': 'req-04b63059-8405-4fa0-82d0-c24cc1d8d399'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:53.109048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] The token cache is being invalidate because project 9a93d2fe75fb4e43b9fd147fd5c8d1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:53.109385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:53.110012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-04b63059-8405-4fa0-82d0-c24cc1d8d399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:53.111541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 935/1865] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:52 2022] DELETE /identity/v3/projects/9a93d2fe75fb4e43b9fd147fd5c8d1a0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:53.116435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c8e9a-5ba2-42fa-81c1-d10f59442cdb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.116887 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c8e9a-5ba2-42fa-81c1-d10f59442cdb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.116997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c8e9a-5ba2-42fa-81c1-d10f59442cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.134440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-259c8e9a-5ba2-42fa-81c1-d10f59442cdb None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:53.135662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-259c8e9a-5ba2-42fa-81c1-d10f59442cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:53.197257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 931/1866] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:53.205667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40357100-4e93-4de6-ba7a-52cfb19540ad None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.206177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40357100-4e93-4de6-ba7a-52cfb19540ad None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.206536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40357100-4e93-4de6-ba7a-52cfb19540ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.227557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-40357100-4e93-4de6-ba7a-52cfb19540ad None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:53.229198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-40357100-4e93-4de6-ba7a-52cfb19540ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:53.237016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 936/1867] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:53.241452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c8e9a-5ba2-42fa-81c1-d10f59442cdb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:53.251818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c8e9a-5ba2-42fa-81c1-d10f59442cdb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.256421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.257092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.257431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.257711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.258476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:53.258786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.260425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.273201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 932/1868] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:53.277430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40357100-4e93-4de6-ba7a-52cfb19540ad None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:53.287614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40357100-4e93-4de6-ba7a-52cfb19540ad None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.292464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.293165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.293468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.293743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.294440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.296535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.325184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 937/1869] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:53 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:53.329549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:53.339996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ef9f6-b4d8-4c2d-b03e-8f16affe256c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.401019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.401962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.402252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.402779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.403483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.406042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.413997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:53.427942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 933/1870] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:53 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:53.432114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:53.442802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a09a7-39b0-48e0-a44b-848ae76f4c7b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.447566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.448223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.448611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.448888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.449728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.451323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.462189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 938/1871] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:53.466085 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:53.476458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4dbd3-415d-496a-ba2b-85454d7f4dc8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.481276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f083d9-b0ed-470a-ae44-f08707dcfce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.481981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f083d9-b0ed-470a-ae44-f08707dcfce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.482318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f083d9-b0ed-470a-ae44-f08707dcfce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.482648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f083d9-b0ed-470a-ae44-f08707dcfce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.483577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f083d9-b0ed-470a-ae44-f08707dcfce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.485234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f083d9-b0ed-470a-ae44-f08707dcfce3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.495108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 934/1872] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:53.499774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:53.509704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841cc60-66c0-47f0-9347-d962332a9864 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.515812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.516714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.517002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.517336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] PATH_INFO: `/v3/projects/e3c8cad3c46447d8865c10f119f1d507/users/7899e1d4b0f848a596ab0533c0a46941/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.520883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:53.534158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c8cad3c46447d8865c10f119f1d507, user_id=7899e1d4b0f848a596ab0533c0a46941, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.536799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.551003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 939/1873] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:53 2022] PUT /identity/v3/projects/e3c8cad3c46447d8865c10f119f1d507/users/7899e1d4b0f848a596ab0533c0a46941/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:53.556315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35bab1-f389-4c61-ac27-9792728d04bb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.556710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35bab1-f389-4c61-ac27-9792728d04bb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.556944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35bab1-f389-4c61-ac27-9792728d04bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.575290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ae35bab1-f389-4c61-ac27-9792728d04bb None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:53.577136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae35bab1-f389-4c61-ac27-9792728d04bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:53.637288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 935/1874] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:53.649902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:53.651603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:53.709773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53642a83-c1b5-4d7e-a501-35609b8c88c2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:53.713397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:53.714046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:53.714329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:53.714606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:53.723598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:53.725031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:53.734292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 940/1875] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:54.594894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e1b4-8368-4c4e-8e4d-b4d4ef7b268c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:54.595416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e1b4-8368-4c4e-8e4d-b4d4ef7b268c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:54.595493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e1b4-8368-4c4e-8e4d-b4d4ef7b268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:54.648367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-05d0e1b4-8368-4c4e-8e4d-b4d4ef7b268c None None] MFA Rules not processed for user `7899e1d4b0f848a596ab0533c0a46941`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:54.649627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05d0e1b4-8368-4c4e-8e4d-b4d4ef7b268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:54.731393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 936/1876] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:54.736123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:54.746173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e01d9a-1a4c-41e4-8619-270576b0d10d service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:54.750971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:54.751721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:54.752077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:54.752421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:54.753241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:54.755666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:54.786006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 941/1877] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:54 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:56:54.790329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0e1b4-8368-4c4e-8e4d-b4d4ef7b268c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:54.802289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0e1b4-8368-4c4e-8e4d-b4d4ef7b268c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:54.859230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:54.859910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:54.860195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:54.860473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:54.861175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:54.863115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:54.879742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:54.885686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 937/1878] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:54 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:54.890193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:54.900590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c23a08-96e4-4f9f-a165-af5987e6a985 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:54.907480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:54.908164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:54.908460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:54.908739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:54.909503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:54.911108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:54.922203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 942/1879] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:54.926480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:54.936954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d6b5-75f8-45a2-913c-20495bd2e7f0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:54.941553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:54.942431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:54.942713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:54.942985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] PATH_INFO: `/v3/projects/27d5a024795244de87a524e2f594130e/users/95b8c59bf6444bf89f0c73f47120cfea/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:54.957974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d5a024795244de87a524e2f594130e, user_id=95b8c59bf6444bf89f0c73f47120cfea, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:54.960604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:54.974111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 938/1880] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:54 2022] PUT /identity/v3/projects/27d5a024795244de87a524e2f594130e/users/95b8c59bf6444bf89f0c73f47120cfea/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:54.978005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:54.988532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254a334-b47c-4edd-86c6-c6924b508161 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:55.046194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aea6e-9afb-463b-9050-4dbd31e1fed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:55.046864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aea6e-9afb-463b-9050-4dbd31e1fed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:55.047155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aea6e-9afb-463b-9050-4dbd31e1fed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:55.047435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aea6e-9afb-463b-9050-4dbd31e1fed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:55.048211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aea6e-9afb-463b-9050-4dbd31e1fed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:55.049916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aea6e-9afb-463b-9050-4dbd31e1fed6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:55.059551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 943/1881] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:55.064284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:55.074396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980ec6c-f251-4063-90d5-7b85eb8f8bfd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:55.079664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:55.080594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:55.081123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:55.081123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] PATH_INFO: `/v3/projects/27d5a024795244de87a524e2f594130e/users/95b8c59bf6444bf89f0c73f47120cfea/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:55.083128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d5a024795244de87a524e2f594130e, user_id=95b8c59bf6444bf89f0c73f47120cfea, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:55.085808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:55.096213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 939/1882] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:56:55 2022] PUT /identity/v3/projects/27d5a024795244de87a524e2f594130e/users/95b8c59bf6444bf89f0c73f47120cfea/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:56.432987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:56.433548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:56.433724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:56.473317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] MFA Rules not processed for user `95b8c59bf6444bf89f0c73f47120cfea`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:56.475735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:56.544123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:56.584329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 944/1883] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:56.595924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:56.597790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:56.670621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a07d6-db11-4a58-835c-be148b57cdd6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:56.674726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21df022-3e61-4fae-907a-7c12d4a8ed9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:56.675832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df022-3e61-4fae-907a-7c12d4a8ed9e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:56.676302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df022-3e61-4fae-907a-7c12d4a8ed9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:56.676740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df022-3e61-4fae-907a-7c12d4a8ed9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:56.739386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21df022-3e61-4fae-907a-7c12d4a8ed9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:56.740959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21df022-3e61-4fae-907a-7c12d4a8ed9e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:56.753342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 940/1884] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:57.286694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:57.288438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:57.354083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd5207-d877-4e94-9da1-6f8ddc93c0b2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:57.359623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:57.360512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:57.360828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:57.361106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] PATH_INFO: `/v3/users/e81be9a1e3334542beda9c1559a95d24` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:57.362340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81be9a1e3334542beda9c1559a95d24)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:57.364309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:57.423350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81be9a1e3334542beda9c1559a95d24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:57.438199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81be9a1e3334542beda9c1559a95d24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:57.439391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81be9a1e3334542beda9c1559a95d24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:57.440983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81be9a1e3334542beda9c1559a95d24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:57.449389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81be9a1e3334542beda9c1559a95d24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:57.462036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 945/1885] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:57 2022] DELETE /identity/v3/users/e81be9a1e3334542beda9c1559a95d24 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:57.466548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:57.467345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:57.467651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:57.488098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:57.489337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:57.573670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 941/1886] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:56:57.584316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:57.585861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:57.665268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f503703-48fd-4ea9-9c8f-b85c8bc3bfa2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:57.676877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:57.677907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:57.678195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:57.678476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:57.694654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:57.696263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:57.702269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:57.709730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 946/1887] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:58.143779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:58.145336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:58.175275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:58.216571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:58.218384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:58.230649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790191d1-87aa-479e-ae16-ef2219321c9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:58.234872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:58.235570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:58.235863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:58.236125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:58.238013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:58.256964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eef09-8ff0-41d4-a5c7-84b91a7753d6 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:58.268606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:58.272307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:58.273058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:58.274794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] PATH_INFO: `/v3/projects/37ce9982a9a24b6dbee6407da433b2d1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:58.276012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ce9982a9a24b6dbee6407da433b2d1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:58.284388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:58.359663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:58.361503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:58.371484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ce9982a9a24b6dbee6407da433b2d1', 'request_id': 'req-a946e5b8-51b0-4281-9606-45d9612fdb25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.380446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ce9982a9a24b6dbee6407da433b2d1', 'request_id': 'req-a946e5b8-51b0-4281-9606-45d9612fdb25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.391061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ce9982a9a24b6dbee6407da433b2d1', 'request_id': 'req-a946e5b8-51b0-4281-9606-45d9612fdb25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.401561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 942/1888] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:56:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:58.414648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] The token cache is being invalidate because project 37ce9982a9a24b6dbee6407da433b2d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:58.415413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.417141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.420554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 947/1889] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:58 2022] DELETE /identity/v3/projects/37ce9982a9a24b6dbee6407da433b2d1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:58.424698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:58.439717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c1f36-a409-4758-aea3-2e7259cea238 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:58.521076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:58.522009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:58.522296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:58.522576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] PATH_INFO: `/v3/users/8d226e4fe3e64f768f3a98de6b2ad5eb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:58.523876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d226e4fe3e64f768f3a98de6b2ad5eb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:58.526114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:58.604906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d226e4fe3e64f768f3a98de6b2ad5eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.619600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d226e4fe3e64f768f3a98de6b2ad5eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.657760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:58.660023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d226e4fe3e64f768f3a98de6b2ad5eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.661266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d226e4fe3e64f768f3a98de6b2ad5eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.662000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5bee5c0f-8e19-4bc2-9422-b7d807c08681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d226e4fe3e64f768f3a98de6b2ad5eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:58.698806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 943/1890] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:56:58 2022] DELETE /identity/v3/users/8d226e4fe3e64f768f3a98de6b2ad5eb => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:59.300631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:59.302307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:59.324733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946e5b8-51b0-4281-9606-45d9612fdb25 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:59.395702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:59.396591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:59.396875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:59.397186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] PATH_INFO: `/v3/projects/c1a8c3ae41a349d4beadf3c3563a1bcb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:59.399062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a8c3ae41a349d4beadf3c3563a1bcb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:59.403266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:59.463815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a8c3ae41a349d4beadf3c3563a1bcb', 'request_id': 'req-1e02009e-9b18-41d1-9f17-0017febd6443'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:59.471886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a8c3ae41a349d4beadf3c3563a1bcb', 'request_id': 'req-1e02009e-9b18-41d1-9f17-0017febd6443'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:59.482514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a8c3ae41a349d4beadf3c3563a1bcb', 'request_id': 'req-1e02009e-9b18-41d1-9f17-0017febd6443'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:59.497887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] The token cache is being invalidate because project c1a8c3ae41a349d4beadf3c3563a1bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:56:59.498280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:59.499544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e02009e-9b18-41d1-9f17-0017febd6443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:56:59.501724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 948/1891] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:56:59 2022] DELETE /identity/v3/projects/c1a8c3ae41a349d4beadf3c3563a1bcb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:56:59.506534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba17f1b-2e6d-48d3-9af3-ad2cc67a7d11 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:59.507011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba17f1b-2e6d-48d3-9af3-ad2cc67a7d11 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:59.507088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba17f1b-2e6d-48d3-9af3-ad2cc67a7d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:59.525865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7ba17f1b-2e6d-48d3-9af3-ad2cc67a7d11 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:59.527270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ba17f1b-2e6d-48d3-9af3-ad2cc67a7d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:59.605068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 944/1892] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:59.613602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510b9ca-7ce7-473c-ac0f-758559260033 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:59.613919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510b9ca-7ce7-473c-ac0f-758559260033 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:59.614167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510b9ca-7ce7-473c-ac0f-758559260033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:59.669111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0510b9ca-7ce7-473c-ac0f-758559260033 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:59.669705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0510b9ca-7ce7-473c-ac0f-758559260033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:59.711411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 949/1893] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:59.712587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba17f1b-2e6d-48d3-9af3-ad2cc67a7d11 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:59.730741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba17f1b-2e6d-48d3-9af3-ad2cc67a7d11 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:59.745187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:59.745637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:59.745921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:59.746195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:59.758659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:56:59.758807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:59.760446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:59.774688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 945/1894] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:56:59.778898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510b9ca-7ce7-473c-ac0f-758559260033 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:59.785537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:59.789420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510b9ca-7ce7-473c-ac0f-758559260033 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:59.795730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:59.796392 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:59.797015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:59.797909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:59.799005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:59.802895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:59.847417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 950/1895] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:56:59 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:59.848807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:59.859745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc960e0a-8b16-45e9-b897-b5935aa8143d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:59.922098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:59.922771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:59.923059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:59.923331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:59.924013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:59.925962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:59.947912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 946/1896] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:56:59 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:56:59.952539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:59.962916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11c81-f777-4b47-afc0-4cefe92c4e17 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:59.968537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:59.969255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:59.969545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:59.969823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:59.970557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:59.972197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:59.990057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 951/1897] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:56:59.990165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:59.998575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b5c9a-4d04-4ce7-8d0e-7f3a6cee1364 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.004033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3e290-91aa-4d44-a7ab-4552886783ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.004712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e290-91aa-4d44-a7ab-4552886783ed admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.004997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e290-91aa-4d44-a7ab-4552886783ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.005319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e290-91aa-4d44-a7ab-4552886783ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.006077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3e290-91aa-4d44-a7ab-4552886783ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.007681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3e290-91aa-4d44-a7ab-4552886783ed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.024282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.027927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 947/1898] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:00.039472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482760d-22ce-4e76-b315-b6ae02203f26 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.048295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.049225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.049530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.049789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] PATH_INFO: `/v3/projects/6dd701ae977f417dbacafb9e376fb489/users/c3f02246cfd2475891a6dc1de26a4b95/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.065404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd701ae977f417dbacafb9e376fb489, user_id=c3f02246cfd2475891a6dc1de26a4b95, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.068065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.083902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 952/1899] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:00 2022] PUT /identity/v3/projects/6dd701ae977f417dbacafb9e376fb489/users/c3f02246cfd2475891a6dc1de26a4b95/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:00.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789807a-47e4-4d15-ad90-4e10128dac93 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.096565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789807a-47e4-4d15-ad90-4e10128dac93 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.096947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789807a-47e4-4d15-ad90-4e10128dac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.122764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b789807a-47e4-4d15-ad90-4e10128dac93 None None] MFA Rules not processed for user `c3f02246cfd2475891a6dc1de26a4b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:00.124009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b789807a-47e4-4d15-ad90-4e10128dac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:00.167674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:00.199109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 948/1900] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:00.204169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.267377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f095f0f-e28a-4c63-a5e7-933e391333b1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.272436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.273262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.273621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.274374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.275154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.277319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.325320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 953/1901] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:00 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:57:00.328415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789807a-47e4-4d15-ad90-4e10128dac93 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.340330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789807a-47e4-4d15-ad90-4e10128dac93 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.403647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.403647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.404088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.404497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.405375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.407576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.440444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 949/1902] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:00 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:00.445611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.462277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c3cf1-5dfe-4c10-b23a-7f2c7652e65a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.470183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.471479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.471921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.472353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.473535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.475497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.495395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 954/1903] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:00.496399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.508248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc73ee3-6faa-4c2f-8b45-4c08a39e6c32 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.514628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.515595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.515886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.516180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] PATH_INFO: `/v3/projects/295cde2e211040ffa7b50852abf2a17c/users/832532b535b94ce2899df77cbd4554af/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.533638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295cde2e211040ffa7b50852abf2a17c, user_id=832532b535b94ce2899df77cbd4554af, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.536910 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.564406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 950/1904] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:00 2022] PUT /identity/v3/projects/295cde2e211040ffa7b50852abf2a17c/users/832532b535b94ce2899df77cbd4554af/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:00.568962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.580148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848bd0-7e36-407b-93cc-ea9ba50aa8e0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.649224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4487e34-2d55-4c1f-9133-2fa0511b07f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.649895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4487e34-2d55-4c1f-9133-2fa0511b07f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.650182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4487e34-2d55-4c1f-9133-2fa0511b07f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.650467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4487e34-2d55-4c1f-9133-2fa0511b07f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.651192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4487e34-2d55-4c1f-9133-2fa0511b07f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.653088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4487e34-2d55-4c1f-9133-2fa0511b07f2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.663420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 955/1905] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:00.667615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.678862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7eb8-bc50-4d6c-ae4d-993ef94cd6c6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.685246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.687589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.688151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.688652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] PATH_INFO: `/v3/projects/295cde2e211040ffa7b50852abf2a17c/users/832532b535b94ce2899df77cbd4554af/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.691126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] RBAC: Authorizing `identity:create_grant(project_id=295cde2e211040ffa7b50852abf2a17c, user_id=832532b535b94ce2899df77cbd4554af, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:00.693979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:00.705327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 951/1906] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:00 2022] PUT /identity/v3/projects/295cde2e211040ffa7b50852abf2a17c/users/832532b535b94ce2899df77cbd4554af/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:00.715830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c0f0-010a-4dfb-a170-0c7c95ce17d2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.716128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c0f0-010a-4dfb-a170-0c7c95ce17d2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.716414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c0f0-010a-4dfb-a170-0c7c95ce17d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:00.743658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-86b4c0f0-010a-4dfb-a170-0c7c95ce17d2 None None] MFA Rules not processed for user `832532b535b94ce2899df77cbd4554af`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:00.745634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-86b4c0f0-010a-4dfb-a170-0c7c95ce17d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:00.751938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:00.833625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 956/1907] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:00.845335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:00.847243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:00.924877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1f77e-a041-4088-b113-eae3cefe701f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:00.929741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:00.930402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:00.931029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:00.931029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:01.030593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:01.047055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:01.048793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:01.059671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 952/1908] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:10.828136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=26 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:11.054097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:13.342048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4c0f0-010a-4dfb-a170-0c7c95ce17d2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:13.351413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4c0f0-010a-4dfb-a170-0c7c95ce17d2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:13.354948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8edf4-5151-4c88-ba33-0d90109960eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:13.355589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8edf4-5151-4c88-ba33-0d90109960eb service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:13.355875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8edf4-5151-4c88-ba33-0d90109960eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:13.356160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8edf4-5151-4c88-ba33-0d90109960eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:13.365053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8edf4-5151-4c88-ba33-0d90109960eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:13.366476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8edf4-5151-4c88-ba33-0d90109960eb service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:13.376452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 957/1909] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:20.926236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:20.928398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:20.999206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdbec0-418d-41f9-853a-7148b9aff492 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:21.004142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:21.005025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:21.005345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:21.005618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] PATH_INFO: `/v3/users/f37ef561d90d4d03bef5aba57c274380` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:21.006819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f37ef561d90d4d03bef5aba57c274380)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:21.008750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:21.059477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f37ef561d90d4d03bef5aba57c274380'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.068606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f37ef561d90d4d03bef5aba57c274380'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.082486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f37ef561d90d4d03bef5aba57c274380'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.083423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f37ef561d90d4d03bef5aba57c274380'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.084557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f37ef561d90d4d03bef5aba57c274380'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.097394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 953/1910] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:20 2022] DELETE /identity/v3/users/f37ef561d90d4d03bef5aba57c274380 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:21.102083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:21.102569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:21.102660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:21.121517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:21.122758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:21.203579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 958/1911] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:21.215145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:21.215145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:21.304510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b62ab-bf5c-49cc-bf20-87acf6ccde07 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:21.308708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:21.309523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:21.309928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:21.310213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:21.320905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:21.326054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:21.349954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 954/1912] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:21.743110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:21.744622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:21.758686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b1b8-a8bc-4284-a524-3a1aa33a0c33 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:21.764738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:21.765726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:21.766076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:21.766422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] PATH_INFO: `/v3/projects/ae9d999268bf42ab85a7f0954f6b951b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:21.767673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae9d999268bf42ab85a7f0954f6b951b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:21.769906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:21.828534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae9d999268bf42ab85a7f0954f6b951b', 'request_id': 'req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.836759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae9d999268bf42ab85a7f0954f6b951b', 'request_id': 'req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.843418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae9d999268bf42ab85a7f0954f6b951b', 'request_id': 'req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.857498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] The token cache is being invalidate because project ae9d999268bf42ab85a7f0954f6b951b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:21.857778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.858419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.860109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 959/1913] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:21 2022] DELETE /identity/v3/projects/ae9d999268bf42ab85a7f0954f6b951b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:21.864157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:21.875135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026054a-02f1-47db-91c6-e586118ed402 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:21.893692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:21.934205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:21.935654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:21.936064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:21.936350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] PATH_INFO: `/v3/users/eef43e77addc49fa954d9279acf889fa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:21.937589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef43e77addc49fa954d9279acf889fa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:21.939639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:21.985887 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef43e77addc49fa954d9279acf889fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:21.993712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef43e77addc49fa954d9279acf889fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.008453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef43e77addc49fa954d9279acf889fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.009166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef43e77addc49fa954d9279acf889fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.009828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb9d9364-dbcc-4d13-adcc-d6723ac814bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef43e77addc49fa954d9279acf889fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.024044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 955/1914] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:21 2022] DELETE /identity/v3/users/eef43e77addc49fa954d9279acf889fa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:22.301102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:22.303141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:22.320281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99796ae8-5dd8-464d-b4e4-b00c6f5f8c8e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:22.389145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:22.390387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.390731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.391066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] PATH_INFO: `/v3/projects/49d9f457f48b4b52af50f5505ae9483f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.392358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d9f457f48b4b52af50f5505ae9483f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:22.394384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:22.407243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:22.448916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d9f457f48b4b52af50f5505ae9483f', 'request_id': 'req-bda6bd8a-d380-456b-889a-553a06b2963f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.458104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d9f457f48b4b52af50f5505ae9483f', 'request_id': 'req-bda6bd8a-d380-456b-889a-553a06b2963f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.466818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d9f457f48b4b52af50f5505ae9483f', 'request_id': 'req-bda6bd8a-d380-456b-889a-553a06b2963f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.479687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] The token cache is being invalidate because project 49d9f457f48b4b52af50f5505ae9483f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:22.480217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.480915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bda6bd8a-d380-456b-889a-553a06b2963f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:22.482585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 960/1915] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:22 2022] DELETE /identity/v3/projects/49d9f457f48b4b52af50f5505ae9483f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:22.487594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78ed0a-0421-46c1-bb2e-1f057fb10635 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.487972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78ed0a-0421-46c1-bb2e-1f057fb10635 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.488316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78ed0a-0421-46c1-bb2e-1f057fb10635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.506325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-da78ed0a-0421-46c1-bb2e-1f057fb10635 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:22.507569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-da78ed0a-0421-46c1-bb2e-1f057fb10635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:22.571054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 956/1916] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:22.579814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f00b9-67dc-42ca-a1d7-5c95285e5b78 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.580103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f00b9-67dc-42ca-a1d7-5c95285e5b78 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.580721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f00b9-67dc-42ca-a1d7-5c95285e5b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.600061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-850f00b9-67dc-42ca-a1d7-5c95285e5b78 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:22.601739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-850f00b9-67dc-42ca-a1d7-5c95285e5b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:22.610209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 961/1917] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:22.614565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78ed0a-0421-46c1-bb2e-1f057fb10635 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:22.625615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78ed0a-0421-46c1-bb2e-1f057fb10635 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:22.630291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:22.630948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.631229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.631508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.632236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:22.632546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:22.634188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:22.647093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 957/1918] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:22.651541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f00b9-67dc-42ca-a1d7-5c95285e5b78 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:22.661860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f00b9-67dc-42ca-a1d7-5c95285e5b78 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:22.666811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:22.667482 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.667764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.668041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.668867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:22.670824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:22.697674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 962/1919] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:22 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:22.702427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:22.712603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af295a6-110e-48b9-bbb4-99fa557abb4e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:22.741222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:22.773977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:22.774658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.774940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.775212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.776535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:22.778114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:22.800216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 958/1920] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:22 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:22.804176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:22.816187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96715e5f-74ca-467b-8cc7-bbaf9fc7ff3b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:22.821437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:22.822124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.822506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.822736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.823451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:22.825061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:22.836312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 963/1921] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:22.841178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:22.851143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e851e1e-039a-4b3b-8157-488bdfc4d4c4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:22.856277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503a46f-515f-4d85-8716-eb7fba037f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:22.856980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a46f-515f-4d85-8716-eb7fba037f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.857298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a46f-515f-4d85-8716-eb7fba037f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.857578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a46f-515f-4d85-8716-eb7fba037f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.858306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503a46f-515f-4d85-8716-eb7fba037f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:22.859913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503a46f-515f-4d85-8716-eb7fba037f44 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:22.870606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 959/1922] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:22.875239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:22.885095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd02a1e-4a40-4682-9542-e9f2f7a150f3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:22.890273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:22.891168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.891450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.891723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] PATH_INFO: `/v3/projects/d9fb0363ca6c47998f3dab4e126b2927/users/a318ba0bf0ab42c39cd7504225b2615f/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.906416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9fb0363ca6c47998f3dab4e126b2927, user_id=a318ba0bf0ab42c39cd7504225b2615f, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:22.909103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:22.921967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 964/1923] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:22 2022] PUT /identity/v3/projects/d9fb0363ca6c47998f3dab4e126b2927/users/a318ba0bf0ab42c39cd7504225b2615f/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:22.932120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c762ed-f34b-407d-b2e1-c3242314be26 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:22.932415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c762ed-f34b-407d-b2e1-c3242314be26 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:22.932696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c762ed-f34b-407d-b2e1-c3242314be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:22.961500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f7c762ed-f34b-407d-b2e1-c3242314be26 None None] MFA Rules not processed for user `a318ba0bf0ab42c39cd7504225b2615f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:22.962727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7c762ed-f34b-407d-b2e1-c3242314be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:23.032536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 960/1924] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:23.037247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:23.099948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1413f-5a67-450d-80ed-b83362d95e46 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:23.104817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:23.105644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.106051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.106329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.107540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:23.109452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:23.134298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 965/1925] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:23 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:23.138284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c762ed-f34b-407d-b2e1-c3242314be26 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:23.148791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c762ed-f34b-407d-b2e1-c3242314be26 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:23.208104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:23.208717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.208996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.209305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.210003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:23.211942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:23.237177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 961/1926] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:23 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:23.238306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:23.249803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360c1c0-c42b-40f2-86b8-a94d19360504 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:23.254600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:23.255308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.255595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.255876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.256608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:23.258397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:23.269120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 966/1927] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:23.273412 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:23.283666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb102d0-a18c-4a14-a518-d2d753dcbdd2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:23.288215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:23.289164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.289457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.289737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] PATH_INFO: `/v3/projects/6304136e651949d3be67c3ef733875b9/users/a9340e0f5ca24061a866eeafe5e87ff2/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.305730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6304136e651949d3be67c3ef733875b9, user_id=a9340e0f5ca24061a866eeafe5e87ff2, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:23.308420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:23.322391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 962/1928] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:23 2022] PUT /identity/v3/projects/6304136e651949d3be67c3ef733875b9/users/a9340e0f5ca24061a866eeafe5e87ff2/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:23.326561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:23.336396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17471c95-14d2-4456-a175-a9348bb7cce4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:23.344078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:23.395275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf6acb-6f71-423b-a316-7b9fdd9c7ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:23.395952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf6acb-6f71-423b-a316-7b9fdd9c7ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.396239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf6acb-6f71-423b-a316-7b9fdd9c7ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.396521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf6acb-6f71-423b-a316-7b9fdd9c7ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.397282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf6acb-6f71-423b-a316-7b9fdd9c7ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:23.398893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf6acb-6f71-423b-a316-7b9fdd9c7ef1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:23.409339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 967/1929] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:23.413617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:23.419028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:23.426269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef3d41-7230-405c-9dd1-6c1a906ef3c3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:23.431105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:23.431993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.432273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.432549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] PATH_INFO: `/v3/projects/6304136e651949d3be67c3ef733875b9/users/a9340e0f5ca24061a866eeafe5e87ff2/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.434516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6304136e651949d3be67c3ef733875b9, user_id=a9340e0f5ca24061a866eeafe5e87ff2, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:23.437366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:23.447821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 963/1930] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:23 2022] PUT /identity/v3/projects/6304136e651949d3be67c3ef733875b9/users/a9340e0f5ca24061a866eeafe5e87ff2/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:23.458959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1daab-8429-4887-a546-c7f7c5304c53 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.459193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1daab-8429-4887-a546-c7f7c5304c53 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.459480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1daab-8429-4887-a546-c7f7c5304c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.486366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-64b1daab-8429-4887-a546-c7f7c5304c53 None None] MFA Rules not processed for user `a9340e0f5ca24061a866eeafe5e87ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:23.487595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-64b1daab-8429-4887-a546-c7f7c5304c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:23.557268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 968/1931] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:23 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:23.567829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:23.569566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:23.630009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de686b-f446-4b4a-b8f0-94a082258066 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:23.633660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bc787-2742-4d68-a2b2-cd708890a2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:23.634306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bc787-2742-4d68-a2b2-cd708890a2fe service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:23.634585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bc787-2742-4d68-a2b2-cd708890a2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:23.634857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bc787-2742-4d68-a2b2-cd708890a2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:23.644017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bc787-2742-4d68-a2b2-cd708890a2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:23.645477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bc787-2742-4d68-a2b2-cd708890a2fe service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:23.655978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 964/1932] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:23 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:24.994734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1daab-8429-4887-a546-c7f7c5304c53 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:25.057411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1daab-8429-4887-a546-c7f7c5304c53 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:25.061916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:25.062785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:25.063067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:25.063338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] PATH_INFO: `/v3/users/a318ba0bf0ab42c39cd7504225b2615f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:25.064522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a318ba0bf0ab42c39cd7504225b2615f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:25.066975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:25.107911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:25.118919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a318ba0bf0ab42c39cd7504225b2615f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.131311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a318ba0bf0ab42c39cd7504225b2615f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.132013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a318ba0bf0ab42c39cd7504225b2615f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.132637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a318ba0bf0ab42c39cd7504225b2615f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.140141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a318ba0bf0ab42c39cd7504225b2615f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.153896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 969/1933] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:24 2022] DELETE /identity/v3/users/a318ba0bf0ab42c39cd7504225b2615f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:25.158442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:25.158899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:25.158999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:25.179538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:25.180758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:25.246338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 965/1934] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:25.255738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:25.257204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:25.315256 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b8fd1-1dee-4a6b-b43d-03c77535d219 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:25.318905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:25.319550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:25.319836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:25.320108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:25.330387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:25.331866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:25.342493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 970/1935] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:25.714185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:25.715607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:25.733144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a51de-9cf0-48e2-9b5e-9483ad87c2c6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:25.737913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:25.738763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:25.739081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:25.739296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] PATH_INFO: `/v3/projects/d9fb0363ca6c47998f3dab4e126b2927` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:25.740487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9fb0363ca6c47998f3dab4e126b2927)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:25.742433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:25.789758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9fb0363ca6c47998f3dab4e126b2927', 'request_id': 'req-027b7a6d-db00-4f0c-a457-feb03c28c1a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.797616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9fb0363ca6c47998f3dab4e126b2927', 'request_id': 'req-027b7a6d-db00-4f0c-a457-feb03c28c1a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.803907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9fb0363ca6c47998f3dab4e126b2927', 'request_id': 'req-027b7a6d-db00-4f0c-a457-feb03c28c1a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.814646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] The token cache is being invalidate because project d9fb0363ca6c47998f3dab4e126b2927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:25.814932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.815555 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.817109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 966/1936] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:25 2022] DELETE /identity/v3/projects/d9fb0363ca6c47998f3dab4e126b2927 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:25.821382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:25.831525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b756c-6a10-4c99-bc0a-14df17f6548f service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:25.886184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:25.887056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:25.887342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:25.888225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] PATH_INFO: `/v3/users/a9340e0f5ca24061a866eeafe5e87ff2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:25.889450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9340e0f5ca24061a866eeafe5e87ff2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:25.891416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:25.939310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9340e0f5ca24061a866eeafe5e87ff2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.946166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:25.952786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9340e0f5ca24061a866eeafe5e87ff2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.953547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9340e0f5ca24061a866eeafe5e87ff2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.954525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9340e0f5ca24061a866eeafe5e87ff2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.961824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-667dd70c-56b2-4a39-b284-c0287f930297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9340e0f5ca24061a866eeafe5e87ff2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:25.973981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 971/1937] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:25 2022] DELETE /identity/v3/users/a9340e0f5ca24061a866eeafe5e87ff2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:26.242983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:26.244635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:26.258557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7a6d-db00-4f0c-a457-feb03c28c1a5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:26.271972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:26.324757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:26.325573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.325857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.326146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] PATH_INFO: `/v3/projects/6304136e651949d3be67c3ef733875b9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.327398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6304136e651949d3be67c3ef733875b9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:26.329500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:26.379253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6304136e651949d3be67c3ef733875b9', 'request_id': 'req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:26.388050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6304136e651949d3be67c3ef733875b9', 'request_id': 'req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:26.394713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6304136e651949d3be67c3ef733875b9', 'request_id': 'req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:26.410386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] The token cache is being invalidate because project 6304136e651949d3be67c3ef733875b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:26.410678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:26.411620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0823648a-d3fd-4d60-b111-ff9bfbeb8b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:26.413201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 967/1938] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:26 2022] DELETE /identity/v3/projects/6304136e651949d3be67c3ef733875b9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:26.418173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68408f8e-ebf3-4733-a5ae-7fa91ae43a08 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.418654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68408f8e-ebf3-4733-a5ae-7fa91ae43a08 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.418765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68408f8e-ebf3-4733-a5ae-7fa91ae43a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.437826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-68408f8e-ebf3-4733-a5ae-7fa91ae43a08 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:26.439032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68408f8e-ebf3-4733-a5ae-7fa91ae43a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:26.497819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 972/1939] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:26.505363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cb6-eec5-4d86-b456-f4b14685cb13 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.505653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cb6-eec5-4d86-b456-f4b14685cb13 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.505943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cb6-eec5-4d86-b456-f4b14685cb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.524654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d81b1cb6-eec5-4d86-b456-f4b14685cb13 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:26.525886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d81b1cb6-eec5-4d86-b456-f4b14685cb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:26.534574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 968/1940] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:26.538894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68408f8e-ebf3-4733-a5ae-7fa91ae43a08 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:26.549510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68408f8e-ebf3-4733-a5ae-7fa91ae43a08 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:26.554452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:26.555123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.555408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.555686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.556418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:26.556728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:26.558363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:26.573191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 973/1941] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:26.575242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1cb6-eec5-4d86-b456-f4b14685cb13 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:26.584679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1cb6-eec5-4d86-b456-f4b14685cb13 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:26.589485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:26.590146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.590426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.591222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.591929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:26.593878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:26.622299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 969/1942] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:26 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:26.626318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:26.636173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df9a0-f548-4034-b873-1b4cca576bdf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:26.711076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:26.712241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.712717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.713086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.714131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:26.717086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:26.741325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 974/1943] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:26 2022] POST /identity/v3/users => generated 281 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:26.743320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:26.753696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a144-52bb-440a-97b0-4e8605dbd8bf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:26.758354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:26.759020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.759310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.759584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.760321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:26.761957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:26.772957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 970/1944] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:26.777342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:26.787702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a20a6-4e23-4c7f-a175-b192ccddca0e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:26.792332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a954cdd-57f3-43ae-b42c-70a46ef52ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:26.793000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a954cdd-57f3-43ae-b42c-70a46ef52ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.793322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a954cdd-57f3-43ae-b42c-70a46ef52ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.793606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a954cdd-57f3-43ae-b42c-70a46ef52ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.794947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a954cdd-57f3-43ae-b42c-70a46ef52ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:26.796536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a954cdd-57f3-43ae-b42c-70a46ef52ef0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:26.806922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 975/1945] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:26.811239 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:26.821012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe163b-f85d-442c-9619-ac9fb3a3d653 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:26.826442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:26.827269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.827550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.827827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] PATH_INFO: `/v3/projects/23d03ce99d4f46938e713b5530a0337d/users/c2ddbdd6e47e42f5a514ca0c17314950/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.843542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d03ce99d4f46938e713b5530a0337d, user_id=c2ddbdd6e47e42f5a514ca0c17314950, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:26.845896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:26.859874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 971/1946] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:26 2022] PUT /identity/v3/projects/23d03ce99d4f46938e713b5530a0337d/users/c2ddbdd6e47e42f5a514ca0c17314950/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:26.871499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820c7f7-c1ec-421e-83ac-de6def1f1f19 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:26.871793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820c7f7-c1ec-421e-83ac-de6def1f1f19 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:26.872081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820c7f7-c1ec-421e-83ac-de6def1f1f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:26.898877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c820c7f7-c1ec-421e-83ac-de6def1f1f19 None None] MFA Rules not processed for user `c2ddbdd6e47e42f5a514ca0c17314950`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:26.900125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c820c7f7-c1ec-421e-83ac-de6def1f1f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:26.930692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:26.968838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 976/1947] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:26.999685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:27.001421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:27.048690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:27.061736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b379213-5033-45de-8099-a5018d27f8c4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:27.065417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6e33f-4706-457a-8771-a39f4be71c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:27.066093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e33f-4706-457a-8771-a39f4be71c0f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:27.066376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e33f-4706-457a-8771-a39f4be71c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:27.066652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e33f-4706-457a-8771-a39f4be71c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:27.074988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6e33f-4706-457a-8771-a39f4be71c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:27.076945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6e33f-4706-457a-8771-a39f4be71c0f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:27.085506 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 972/1948] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:27.099846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820c7f7-c1ec-421e-83ac-de6def1f1f19 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:27.161473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820c7f7-c1ec-421e-83ac-de6def1f1f19 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:27.166194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:27.167336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:27.167627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:27.167905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] PATH_INFO: `/v3/users/c2ddbdd6e47e42f5a514ca0c17314950` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:27.169146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ddbdd6e47e42f5a514ca0c17314950)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:27.171110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:27.219163 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ddbdd6e47e42f5a514ca0c17314950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.231209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ddbdd6e47e42f5a514ca0c17314950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.231944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ddbdd6e47e42f5a514ca0c17314950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.232556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ddbdd6e47e42f5a514ca0c17314950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.239446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ddbdd6e47e42f5a514ca0c17314950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.252155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 977/1949] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:27 2022] DELETE /identity/v3/users/c2ddbdd6e47e42f5a514ca0c17314950 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:27.256601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:27.257055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:27.257190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:27.274432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:27.275611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:27.333683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 973/1950] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:27.343276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:27.344694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:27.404676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:27.409142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a7f3d-b97e-442b-8fea-23303fa50e68 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:27.414355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea675cb-dd8c-44df-9d30-e812bf162bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:27.415264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea675cb-dd8c-44df-9d30-e812bf162bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:27.415726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea675cb-dd8c-44df-9d30-e812bf162bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:27.416568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea675cb-dd8c-44df-9d30-e812bf162bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:27.426636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea675cb-dd8c-44df-9d30-e812bf162bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:27.428255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea675cb-dd8c-44df-9d30-e812bf162bf2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:27.438737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 978/1951] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:27.740285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:27.742211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:27.765806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bcfd1-b7bc-4f9c-b7d4-00a10b262b4f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:27.772483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:27.773394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:27.773675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:27.774083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] PATH_INFO: `/v3/projects/23d03ce99d4f46938e713b5530a0337d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:27.775466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d03ce99d4f46938e713b5530a0337d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:27.777684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:27.831364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d03ce99d4f46938e713b5530a0337d', 'request_id': 'req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.839504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d03ce99d4f46938e713b5530a0337d', 'request_id': 'req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.846780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d03ce99d4f46938e713b5530a0337d', 'request_id': 'req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.857337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] The token cache is being invalidate because project 23d03ce99d4f46938e713b5530a0337d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:27.857623 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.858263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c9c07ee-19a1-4da1-8db1-7c34b1f59fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:27.859791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 974/1952] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:27 2022] DELETE /identity/v3/projects/23d03ce99d4f46938e713b5530a0337d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:27.864517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325651b-5110-48f9-9d5a-b68ee7697949 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:27.864810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325651b-5110-48f9-9d5a-b68ee7697949 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:27.865139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325651b-5110-48f9-9d5a-b68ee7697949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:27.883373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8325651b-5110-48f9-9d5a-b68ee7697949 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:27.884569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8325651b-5110-48f9-9d5a-b68ee7697949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:27.948645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 979/1953] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:27.956192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f88c7-2bab-4f56-986d-3f3c306a85c6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:27.956480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f88c7-2bab-4f56-986d-3f3c306a85c6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:27.956755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f88c7-2bab-4f56-986d-3f3c306a85c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:27.976434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f65f88c7-2bab-4f56-986d-3f3c306a85c6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:27.977849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f65f88c7-2bab-4f56-986d-3f3c306a85c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:27.986941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 975/1954] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:27.991007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325651b-5110-48f9-9d5a-b68ee7697949 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.000690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325651b-5110-48f9-9d5a-b68ee7697949 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.006048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.006861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.007250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.007628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.008471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:28.008881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.010782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.023969 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 980/1955] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:28.029399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f88c7-2bab-4f56-986d-3f3c306a85c6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.042507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f88c7-2bab-4f56-986d-3f3c306a85c6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.062310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.062995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.070601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.070884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.071679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.073642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.109620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 976/1956] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:28 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:28.112844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.124101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a4952-1ab1-4c7f-b553-1adde9dcafe8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.189790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.190800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.190951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.191235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.192005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.194803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.217682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 981/1957] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:28 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:28.221695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.233464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827df99-a90f-4c5b-a08f-fa4adcab93e9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.238160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.238863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.239109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.239855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.240113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.241754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.253742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 977/1958] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:28.257662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.268620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013429e8-6bac-4cb7-9dbf-8f1a3cf5c3cb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.274280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7146f0-f14d-4ba0-9811-8208ff85ebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.275070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7146f0-f14d-4ba0-9811-8208ff85ebf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.275359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7146f0-f14d-4ba0-9811-8208ff85ebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.275632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7146f0-f14d-4ba0-9811-8208ff85ebf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.276355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7146f0-f14d-4ba0-9811-8208ff85ebf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.278773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7146f0-f14d-4ba0-9811-8208ff85ebf8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.289720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 982/1959] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:28.294616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.304756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5896eeb-edcf-47ad-9233-db11c10c0da1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.311897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.312802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.313091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.313400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] PATH_INFO: `/v3/projects/784eb73dfd5b4fa0b9b3cc62f5e1ec88/users/16d986c63a634197a85fd2da4b4256ab/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.330066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=784eb73dfd5b4fa0b9b3cc62f5e1ec88, user_id=16d986c63a634197a85fd2da4b4256ab, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.332726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.346898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 978/1960] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:28 2022] PUT /identity/v3/projects/784eb73dfd5b4fa0b9b3cc62f5e1ec88/users/16d986c63a634197a85fd2da4b4256ab/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:28.357123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca3b2-f6c2-4a1a-811d-f33004eb9b1d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.357460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca3b2-f6c2-4a1a-811d-f33004eb9b1d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.357746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca3b2-f6c2-4a1a-811d-f33004eb9b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.386001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-df5ca3b2-f6c2-4a1a-811d-f33004eb9b1d None None] MFA Rules not processed for user `16d986c63a634197a85fd2da4b4256ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:28.387233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-df5ca3b2-f6c2-4a1a-811d-f33004eb9b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:28.435513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:28.459669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 983/1961] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:28.464125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.470742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:28.538405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc411c-0262-42c7-ba5e-ca783132d6e5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.543475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.544178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.544464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.544738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.545500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.547490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.579927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 979/1962] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:28 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:28.581208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ca3b2-f6c2-4a1a-811d-f33004eb9b1d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.602276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ca3b2-f6c2-4a1a-811d-f33004eb9b1d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.702586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.703286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.713481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.715985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.716829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.719132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.757600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 984/1963] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:28 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:57:28.761313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.775594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b78eb-1c65-468a-a94a-91338f094f50 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.802574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.810776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.811951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.813538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.814458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.819457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.855871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 980/1964] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:28.856911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.874231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9845-bb02-4f58-853a-3e23cdf1471c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:28.884449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:28.886859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:28.886953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:28.887247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] PATH_INFO: `/v3/projects/a2567a6fc3dd4157a9808bed1752e87f/users/ae1a08ee55c8406f8f291c1ad92bf6f8/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:28.909896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2567a6fc3dd4157a9808bed1752e87f, user_id=ae1a08ee55c8406f8f291c1ad92bf6f8, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:28.913602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:28.940547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 985/1965] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:28 2022] PUT /identity/v3/projects/a2567a6fc3dd4157a9808bed1752e87f/users/ae1a08ee55c8406f8f291c1ad92bf6f8/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:28.944984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:28.958215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a40f82-e20a-4341-8229-01da3a1cf51c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:29.059141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4605d-fc01-4ebe-aede-ed0c0182af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:29.060195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4605d-fc01-4ebe-aede-ed0c0182af06 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:29.060492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4605d-fc01-4ebe-aede-ed0c0182af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:29.060772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4605d-fc01-4ebe-aede-ed0c0182af06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:29.061869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4605d-fc01-4ebe-aede-ed0c0182af06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:29.063316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4605d-fc01-4ebe-aede-ed0c0182af06 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:29.078121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 981/1966] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:29.082041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:29.102323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2ed3c-9fb0-4aa8-ae71-54cba409d471 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:29.107693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:29.108672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:29.109103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:29.109416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] PATH_INFO: `/v3/projects/a2567a6fc3dd4157a9808bed1752e87f/users/ae1a08ee55c8406f8f291c1ad92bf6f8/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:29.111418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2567a6fc3dd4157a9808bed1752e87f, user_id=ae1a08ee55c8406f8f291c1ad92bf6f8, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:29.114713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:29.128443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 986/1967] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:29 2022] PUT /identity/v3/projects/a2567a6fc3dd4157a9808bed1752e87f/users/ae1a08ee55c8406f8f291c1ad92bf6f8/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:29.141078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c85f9-cc7b-405b-822e-df3a5ef081c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:29.141499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c85f9-cc7b-405b-822e-df3a5ef081c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:29.141853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c85f9-cc7b-405b-822e-df3a5ef081c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:29.183587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0c3c85f9-cc7b-405b-822e-df3a5ef081c3 None None] MFA Rules not processed for user `ae1a08ee55c8406f8f291c1ad92bf6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:29.194431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c3c85f9-cc7b-405b-822e-df3a5ef081c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:29.281665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:29.287201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 982/1968] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:29.301866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:29.306894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:29.392731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461448b-411d-4eab-ad2c-126eb3e62a3f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:29.398663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8809d4-50c9-4a2c-bc84-3b922ab09eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:29.399332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8809d4-50c9-4a2c-bc84-3b922ab09eff service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:29.399683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8809d4-50c9-4a2c-bc84-3b922ab09eff service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:29.402651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8809d4-50c9-4a2c-bc84-3b922ab09eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:29.495028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:29.499454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8809d4-50c9-4a2c-bc84-3b922ab09eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:29.502305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8809d4-50c9-4a2c-bc84-3b922ab09eff service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:29.512169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 987/1969] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:38.523515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c85f9-cc7b-405b-822e-df3a5ef081c3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:38.585867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c85f9-cc7b-405b-822e-df3a5ef081c3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:38.592642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:38.594079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:38.594369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:38.594681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] PATH_INFO: `/v3/users/16d986c63a634197a85fd2da4b4256ab` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:38.595926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d986c63a634197a85fd2da4b4256ab)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:38.597944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:38.647194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d986c63a634197a85fd2da4b4256ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.651720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec006bf-8970-46f8-bed9-ac013034953d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:38.652487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec006bf-8970-46f8-bed9-ac013034953d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:38.653475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec006bf-8970-46f8-bed9-ac013034953d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:38.655586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 988/1970] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 13:57:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:57:38.656785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d986c63a634197a85fd2da4b4256ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.663763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:38.664050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:38.664456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:38.671915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d986c63a634197a85fd2da4b4256ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.672662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d986c63a634197a85fd2da4b4256ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.673323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97a61b9c-7f5c-4e53-9882-cc770f6539ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d986c63a634197a85fd2da4b4256ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.683736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] MFA Rules not processed for user `997348872c3241f4a14c6b2fe9b66211`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:38.684956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:38.687319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 983/1971] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:38 2022] DELETE /identity/v3/users/16d986c63a634197a85fd2da4b4256ab => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:38.691464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:38.691732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:38.692012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:38.708891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:38.710290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:38.773055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 989/1972] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 13:57:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:38.774159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:38.775394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:38.821313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 984/1973] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:38.822576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:38.825293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:38.846634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe8af1-6638-4ec5-957b-89a655fa6db0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:38.853039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:38.853237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21fdd-32d3-44ce-909f-fe3f427864c9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:38.853844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:38.854289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:38.854693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:38.858163 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:38.859031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:38.859314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:38.859589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] PATH_INFO: `/v3/users/7899e1d4b0f848a596ab0533c0a46941` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:38.861529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7899e1d4b0f848a596ab0533c0a46941)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:38.864337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:38.865079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:38.866704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:38.877347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 985/1974] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:38.878283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:38.879756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:38.893078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:38.927710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7899e1d4b0f848a596ab0533c0a46941'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.944522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7899e1d4b0f848a596ab0533c0a46941'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.945855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7899e1d4b0f848a596ab0533c0a46941'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.946567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7899e1d4b0f848a596ab0533c0a46941'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:38.968806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7899e1d4b0f848a596ab0533c0a46941'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.000975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 990/1975] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:38 2022] DELETE /identity/v3/users/7899e1d4b0f848a596ab0533c0a46941 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:39.002461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e02618-5a2a-429a-931b-d4727f8c7815 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:39.062836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:39.063498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.063810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.064089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.066211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:39.131290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:39.132727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:39.143575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 986/1976] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:39.212085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:39.213899 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:39.232656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa684c8a-8b58-461f-a960-64e6adce4ff3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:39.238204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:39.239265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.239561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.239905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] PATH_INFO: `/v3/projects/e3c8cad3c46447d8865c10f119f1d507` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.241203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c8cad3c46447d8865c10f119f1d507)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:39.243213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:39.246811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:39.318113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c8cad3c46447d8865c10f119f1d507', 'request_id': 'req-ffe82a2a-d862-4581-93c6-1e146ff59e1c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.324802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c8cad3c46447d8865c10f119f1d507', 'request_id': 'req-ffe82a2a-d862-4581-93c6-1e146ff59e1c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.330693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c8cad3c46447d8865c10f119f1d507', 'request_id': 'req-ffe82a2a-d862-4581-93c6-1e146ff59e1c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.344440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] The token cache is being invalidate because project e3c8cad3c46447d8865c10f119f1d507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:39.344729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.345771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.347041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 991/1977] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:39 2022] DELETE /identity/v3/projects/e3c8cad3c46447d8865c10f119f1d507 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:39.350979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:39.361180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7572b05-08f4-4e10-82d5-c92d0193d6f5 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:39.436320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:39.437474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.437825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.438141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] PATH_INFO: `/v3/users/95b8c59bf6444bf89f0c73f47120cfea` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.439356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95b8c59bf6444bf89f0c73f47120cfea)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:39.441390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:39.457419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:39.468329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:39.469934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:39.492970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe82a2a-d862-4581-93c6-1e146ff59e1c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:39.499711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:39.500554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.500833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.501221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] PATH_INFO: `/v3/projects/784eb73dfd5b4fa0b9b3cc62f5e1ec88` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.502485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=784eb73dfd5b4fa0b9b3cc62f5e1ec88)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:39.503386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95b8c59bf6444bf89f0c73f47120cfea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.504457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:39.511702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95b8c59bf6444bf89f0c73f47120cfea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.525649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95b8c59bf6444bf89f0c73f47120cfea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.526719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95b8c59bf6444bf89f0c73f47120cfea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.527341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95b8c59bf6444bf89f0c73f47120cfea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.541294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 987/1978] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:39 2022] DELETE /identity/v3/users/95b8c59bf6444bf89f0c73f47120cfea => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:39.554995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784eb73dfd5b4fa0b9b3cc62f5e1ec88', 'request_id': 'req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.561388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784eb73dfd5b4fa0b9b3cc62f5e1ec88', 'request_id': 'req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.567319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784eb73dfd5b4fa0b9b3cc62f5e1ec88', 'request_id': 'req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.581592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] The token cache is being invalidate because project 784eb73dfd5b4fa0b9b3cc62f5e1ec88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:39.582185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.582526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.584143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 992/1979] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:39 2022] DELETE /identity/v3/projects/784eb73dfd5b4fa0b9b3cc62f5e1ec88 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:39.589665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:39.591666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:39.658416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb6b16-b458-4a0e-bc5d-898846f1b430 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:39.723362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:39.724544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:39.725356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:39.725587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.725967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.726255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] PATH_INFO: `/v3/users/ae1a08ee55c8406f8f291c1ad92bf6f8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.727498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1a08ee55c8406f8f291c1ad92bf6f8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:39.729470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:39.738691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234ff54-5b2d-479c-a2e7-8fa7c418eec7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:39.743718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:39.744610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.745203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.745203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] PATH_INFO: `/v3/projects/27d5a024795244de87a524e2f594130e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.746364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d5a024795244de87a524e2f594130e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:39.748309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:39.784014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:39.790234 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1a08ee55c8406f8f291c1ad92bf6f8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.798474 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1a08ee55c8406f8f291c1ad92bf6f8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.805195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d5a024795244de87a524e2f594130e', 'request_id': 'req-96035873-5326-4fbe-8856-324ad0f8dcc5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.810724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1a08ee55c8406f8f291c1ad92bf6f8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.811413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1a08ee55c8406f8f291c1ad92bf6f8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.811674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d5a024795244de87a524e2f594130e', 'request_id': 'req-96035873-5326-4fbe-8856-324ad0f8dcc5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.812033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-46bbf7e2-9652-4ae8-a19e-350e21d5e970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1a08ee55c8406f8f291c1ad92bf6f8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.817342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d5a024795244de87a524e2f594130e', 'request_id': 'req-96035873-5326-4fbe-8856-324ad0f8dcc5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.825647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 988/1980] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:39 2022] DELETE /identity/v3/users/ae1a08ee55c8406f8f291c1ad92bf6f8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:39.833821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] The token cache is being invalidate because project 27d5a024795244de87a524e2f594130e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:39.834123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.834774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96035873-5326-4fbe-8856-324ad0f8dcc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:39.836366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 993/1981] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:39 2022] DELETE /identity/v3/projects/27d5a024795244de87a524e2f594130e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:39.836857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=76,keystone:DELETE=21,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:39.842111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f6fb2-dfb2-4638-807e-526be94972ca None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.842409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f6fb2-dfb2-4638-807e-526be94972ca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.842699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f6fb2-dfb2-4638-807e-526be94972ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.860924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b85f6fb2-dfb2-4638-807e-526be94972ca None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:39.862222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b85f6fb2-dfb2-4638-807e-526be94972ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:39.931634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 989/1982] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:39.939931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b0c2-819e-48e5-bcc2-190f744c73ca None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.940354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b0c2-819e-48e5-bcc2-190f744c73ca None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.940637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b0c2-819e-48e5-bcc2-190f744c73ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.959665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0371b0c2-819e-48e5-bcc2-190f744c73ca None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:39.960886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0371b0c2-819e-48e5-bcc2-190f744c73ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:39.975450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 994/1983] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:39.977332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f6fb2-dfb2-4638-807e-526be94972ca None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:39.987598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f6fb2-dfb2-4638-807e-526be94972ca None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:39.992343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:39.993088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:39.993426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:39.993835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:39.995781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:39.995781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:39.997225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.016819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 990/1984] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:40.018043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371b0c2-819e-48e5-bcc2-190f744c73ca None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.028632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371b0c2-819e-48e5-bcc2-190f744c73ca None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.035110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.035872 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.036232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.036559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.037572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.039461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.092797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 995/1985] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:40.093025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.103340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f806fc-efcc-4592-ab72-5573d793502d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.161942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.162534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.163280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.163667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.164371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.166333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.201189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 991/1986] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:40.205199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.216264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae16f2-6537-4cf0-af0d-a38efd96fb57 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.222152 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.222835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.223122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.223399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.224128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.225853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.238475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 996/1987] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:57:40.238635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.239985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:40.240939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.252737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf706e3-dc6a-4481-a888-f0733506c7f8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.253302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065dc3c-b51c-4a31-9367-5336bffcc896 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.258249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.258928 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.259207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.259494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.259781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.260305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.260717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.261010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.261339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] PATH_INFO: `/v3/projects/a2567a6fc3dd4157a9808bed1752e87f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.262941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.263863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2567a6fc3dd4157a9808bed1752e87f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.274450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 997/1988] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:40.275181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.279309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.294425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba5460-3ede-4367-b0bb-a4b825c0086d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.299780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce84913-1e07-4206-a4de-d734738ad26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.300879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce84913-1e07-4206-a4de-d734738ad26a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.301193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce84913-1e07-4206-a4de-d734738ad26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.301484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce84913-1e07-4206-a4de-d734738ad26a admin admin] PATH_INFO: `/v3/projects/70e1d74aa6364176aced7e3bcdc078c1/users/6fe2c541608f404298577c8090d4f69f/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.323963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce84913-1e07-4206-a4de-d734738ad26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e1d74aa6364176aced7e3bcdc078c1, user_id=6fe2c541608f404298577c8090d4f69f, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.334010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce84913-1e07-4206-a4de-d734738ad26a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.335238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:40.342513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2567a6fc3dd4157a9808bed1752e87f', 'request_id': 'req-2ccfe007-3baa-4932-95a3-17e6d91188bb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:40.350407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 998/1989] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:40 2022] PUT /identity/v3/projects/70e1d74aa6364176aced7e3bcdc078c1/users/6fe2c541608f404298577c8090d4f69f/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:40.352569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2567a6fc3dd4157a9808bed1752e87f', 'request_id': 'req-2ccfe007-3baa-4932-95a3-17e6d91188bb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:40.360362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2567a6fc3dd4157a9808bed1752e87f', 'request_id': 'req-2ccfe007-3baa-4932-95a3-17e6d91188bb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:40.362606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.362932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.363215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.372450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] The token cache is being invalidate because project a2567a6fc3dd4157a9808bed1752e87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:40.372747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:40.373458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ccfe007-3baa-4932-95a3-17e6d91188bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:40.375067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 992/1990] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:40 2022] DELETE /identity/v3/projects/a2567a6fc3dd4157a9808bed1752e87f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:40.380006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9003c8-9adf-43ba-828c-007f4e69e81c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.380327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9003c8-9adf-43ba-828c-007f4e69e81c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.380611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9003c8-9adf-43ba-828c-007f4e69e81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.392049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] MFA Rules not processed for user `6fe2c541608f404298577c8090d4f69f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:40.393438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:40.398416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dd9003c8-9adf-43ba-828c-007f4e69e81c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:40.399599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd9003c8-9adf-43ba-828c-007f4e69e81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:40.467379 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 999/1991] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:40.470996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 993/1992] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:40.471902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.473654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:40.478632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a679f-80dc-4c8e-9ec9-2c267b9b9a72 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.479107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a679f-80dc-4c8e-9ec9-2c267b9b9a72 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.479530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a679f-80dc-4c8e-9ec9-2c267b9b9a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.491080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2dab8-46e2-43aa-8029-e58ec6f8adbb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.495990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.497250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.497534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.497809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.498496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.500507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.501890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3b5a679f-80dc-4c8e-9ec9-2c267b9b9a72 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:40.503110 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b5a679f-80dc-4c8e-9ec9-2c267b9b9a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:40.512058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 994/1993] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:40.516476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a679f-80dc-4c8e-9ec9-2c267b9b9a72 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.526221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1000/1994] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:40.527173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a679f-80dc-4c8e-9ec9-2c267b9b9a72 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.530317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.583764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:40.585291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.585844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.586133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.586462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.587374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:40.587686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.589324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.595628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584ade3-05b7-4908-af0b-4b41a0452ffd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.601321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.602026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.602262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.602546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.606791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 995/1995] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:40.606952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.606952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.607773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.618517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0216-e989-4737-88be-27c5987fcfba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.624419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.625072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.625386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.625673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.626359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.628266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.628413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1001/1996] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:57:40.632051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.642249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26aa4c-539f-435b-8b03-5af12bebecb5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.647208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.648026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.648391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.648626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.649377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.650977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.655782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 996/1997] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:40.659681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.666354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1002/1998] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:57:40.667451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.677728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d22198-9faf-475e-b936-59bfe049373d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.687668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:40.724023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cabd48-9097-4209-85c0-2f53c32838ac admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.728879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.729574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.729854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.730126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.731531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.733509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.735722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.736615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.736893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.737183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] PATH_INFO: `/v3/projects/db8902bb66da4ebe9aa7a7f9832b7aac/users/eaf07abffe9243a49d9948e929659c33/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.752657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8902bb66da4ebe9aa7a7f9832b7aac, user_id=eaf07abffe9243a49d9948e929659c33, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.754137 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 997/1999] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:57:40.755545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.757845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.767873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bc6a-4840-4d0c-8a53-f1acc8da2cd2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.769772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1003/2000] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:40 2022] PUT /identity/v3/projects/db8902bb66da4ebe9aa7a7f9832b7aac/users/eaf07abffe9243a49d9948e929659c33/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:40.773525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.828751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.829467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.829757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.830010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.830725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.832327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.837508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74549b-1cad-4781-a103-d00be95cce34 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.843758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 998/2001] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:57:40.846786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.847487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.847598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.847775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.848154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.848891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.850526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.857496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9a974-b99e-4931-b752-5085b2e7e448 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.864153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1004/2002] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:57:40.864297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.864297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.864297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.864297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.864648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.865425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.865724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.875094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829656ce-be68-4788-9452-44ddb228376d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.876349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 999/2003] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:57:40.879989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccf394-58b0-47b3-86b4-fe97aeef9ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.880152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:40.880865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccf394-58b0-47b3-86b4-fe97aeef9ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.881165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccf394-58b0-47b3-86b4-fe97aeef9ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.881459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccf394-58b0-47b3-86b4-fe97aeef9ba6 admin admin] PATH_INFO: `/v3/projects/db8902bb66da4ebe9aa7a7f9832b7aac/users/eaf07abffe9243a49d9948e929659c33/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.883434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccf394-58b0-47b3-86b4-fe97aeef9ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8902bb66da4ebe9aa7a7f9832b7aac, user_id=eaf07abffe9243a49d9948e929659c33, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.886158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccf394-58b0-47b3-86b4-fe97aeef9ba6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.889621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605b37-1acb-4bf4-8571-16d8ceb3a85b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:40.894143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86c52-687e-489c-b105-abe7ac429766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:40.894992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86c52-687e-489c-b105-abe7ac429766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.895267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86c52-687e-489c-b105-abe7ac429766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.895533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86c52-687e-489c-b105-abe7ac429766 admin admin] PATH_INFO: `/v3/projects/631cff438f1d44458d9def61d2960db4/users/12395b325e06408ea21ac1d503160cd9/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.896475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1005/2004] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:40 2022] PUT /identity/v3/projects/db8902bb66da4ebe9aa7a7f9832b7aac/users/eaf07abffe9243a49d9948e929659c33/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:40.907373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.907653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.907922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.911749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86c52-687e-489c-b105-abe7ac429766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631cff438f1d44458d9def61d2960db4, user_id=12395b325e06408ea21ac1d503160cd9, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:40.914927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86c52-687e-489c-b105-abe7ac429766 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:40.930299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1000/2005] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:40 2022] PUT /identity/v3/projects/631cff438f1d44458d9def61d2960db4/users/12395b325e06408ea21ac1d503160cd9/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:40.939370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] MFA Rules not processed for user `eaf07abffe9243a49d9948e929659c33`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:40.940594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:40.940854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8371323-ffee-405d-909d-63850df52efe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:40.941182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8371323-ffee-405d-909d-63850df52efe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:40.941460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8371323-ffee-405d-909d-63850df52efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:40.966104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:40.968086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a8371323-ffee-405d-909d-63850df52efe None None] MFA Rules not processed for user `12395b325e06408ea21ac1d503160cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:40.969491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a8371323-ffee-405d-909d-63850df52efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:41.013857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1006/2006] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:41.025806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:41.028659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:41.039361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1001/2007] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:41.052107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8371323-ffee-405d-909d-63850df52efe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:41.053794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a8371323-ffee-405d-909d-63850df52efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:41.060554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:41.089974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059ece1-b2b6-481c-b69a-38338a159d74 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:41.094167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70aa1f5-3963-4750-8b47-c69110734052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:41.094838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aa1f5-3963-4750-8b47-c69110734052 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:41.095119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aa1f5-3963-4750-8b47-c69110734052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:41.095503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aa1f5-3963-4750-8b47-c69110734052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:41.114767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8371323-ffee-405d-909d-63850df52efe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:41.118572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:41.119228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:41.119508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:41.119777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:41.128209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:41.130331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:41.140057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1002/2008] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:57:41.153395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70aa1f5-3963-4750-8b47-c69110734052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:41.154840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70aa1f5-3963-4750-8b47-c69110734052 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:41.163896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1007/2009] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:57:45.841269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:45.909686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99902f-e72f-48f1-9a0b-0b22ab828fb3 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:45.914971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:45.916071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:45.916559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:45.917024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] PATH_INFO: `/v3/users/12395b325e06408ea21ac1d503160cd9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:45.918466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] RBAC: Authorizing `identity:delete_user(user_id=12395b325e06408ea21ac1d503160cd9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:45.921367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:45.985276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12395b325e06408ea21ac1d503160cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:45.997853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12395b325e06408ea21ac1d503160cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.014631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12395b325e06408ea21ac1d503160cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.015595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12395b325e06408ea21ac1d503160cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.017018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12395b325e06408ea21ac1d503160cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.040224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1003/2010] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:45 2022] DELETE /identity/v3/users/12395b325e06408ea21ac1d503160cd9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:46.047635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:46.048108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:46.048186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:46.071227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:46.072551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:46.207851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1008/2011] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:46.219609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:46.221501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:46.259870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:46.292020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285e35a-d37a-4316-8c99-a07d1ee959de admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:46.296040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bea279-3b32-4ae8-8ada-0636cbc0e91b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:46.296695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea279-3b32-4ae8-8ada-0636cbc0e91b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:46.296983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea279-3b32-4ae8-8ada-0636cbc0e91b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:46.297298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea279-3b32-4ae8-8ada-0636cbc0e91b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:46.306546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bea279-3b32-4ae8-8ada-0636cbc0e91b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:46.307976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bea279-3b32-4ae8-8ada-0636cbc0e91b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:46.318209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1004/2012] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:46.602804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:46.604442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:46.625270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833e65a-ffbb-4960-9bee-655b7b917922 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:46.638031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:46.639011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:46.639369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:46.639753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] PATH_INFO: `/v3/projects/631cff438f1d44458d9def61d2960db4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:46.641077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631cff438f1d44458d9def61d2960db4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:46.643156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:46.655460 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:46.700522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631cff438f1d44458d9def61d2960db4', 'request_id': 'req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.707212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631cff438f1d44458d9def61d2960db4', 'request_id': 'req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.713235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631cff438f1d44458d9def61d2960db4', 'request_id': 'req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.726568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] The token cache is being invalidate because project 631cff438f1d44458d9def61d2960db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:46.726799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.727486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bb511b91-bdd8-4b67-8269-c0c52c13a1b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:46.728969 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1009/2013] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:46 2022] DELETE /identity/v3/projects/631cff438f1d44458d9def61d2960db4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:46.734375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149af2-1e48-4c33-89e0-2fd9a1c473fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:46.734766 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149af2-1e48-4c33-89e0-2fd9a1c473fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:46.735144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149af2-1e48-4c33-89e0-2fd9a1c473fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:46.757622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-09149af2-1e48-4c33-89e0-2fd9a1c473fe None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:46.759001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-09149af2-1e48-4c33-89e0-2fd9a1c473fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:46.838626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1005/2014] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:46.841326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322eae0c-40f7-455f-9c6c-2bef5f8930f8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:46.842230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322eae0c-40f7-455f-9c6c-2bef5f8930f8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:46.843012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322eae0c-40f7-455f-9c6c-2bef5f8930f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:46.863564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-322eae0c-40f7-455f-9c6c-2bef5f8930f8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:46.864794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-322eae0c-40f7-455f-9c6c-2bef5f8930f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:46.874920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1010/2015] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:46.879009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09149af2-1e48-4c33-89e0-2fd9a1c473fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:46.890267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09149af2-1e48-4c33-89e0-2fd9a1c473fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:46.903731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:46.904516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:46.904895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:46.905320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:46.916390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:46.916390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:46.917453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:46.933206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1006/2016] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:46.934807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322eae0c-40f7-455f-9c6c-2bef5f8930f8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:46.944393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322eae0c-40f7-455f-9c6c-2bef5f8930f8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:46.949089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:46.949779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:46.950095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:46.950376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:46.951069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:46.952996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:46.979224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1011/2017] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:46 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:46.984182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:46.995212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cc02d-2d3f-4b3b-a9a7-3ecd404f050d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:47.052617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:47.053241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:47.053523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:47.053797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:47.054478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:47.056388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:47.075700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1007/2018] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:46 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:57:47.079629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:47.089954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3f96-4d9d-470d-867f-7b3a5a6810c5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:47.095318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:47.095976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:47.096262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:47.096567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:47.097335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:47.098929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:47.109139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1012/2019] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:47.112849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:47.123640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7e1c-f549-4132-a832-5650657241fa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:47.128258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967f3f0-b1b1-46ca-8752-482977489a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:47.128914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967f3f0-b1b1-46ca-8752-482977489a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:47.129240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967f3f0-b1b1-46ca-8752-482977489a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:47.129522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967f3f0-b1b1-46ca-8752-482977489a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:47.130242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967f3f0-b1b1-46ca-8752-482977489a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:47.131837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967f3f0-b1b1-46ca-8752-482977489a16 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:47.142255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1008/2020] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:47.146383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:47.157021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27da5d-f23d-4371-8a19-05460a5a19be admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:47.161705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:47.162669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:47.162955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:47.163235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] PATH_INFO: `/v3/projects/fea63085a6f64135af82f0f671608c54/users/b499deffb3164de396174e39db5e932d/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:47.179882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea63085a6f64135af82f0f671608c54, user_id=b499deffb3164de396174e39db5e932d, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:47.181612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:47.195556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1013/2021] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:47 2022] PUT /identity/v3/projects/fea63085a6f64135af82f0f671608c54/users/b499deffb3164de396174e39db5e932d/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:47.206470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:47.206767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:47.207047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:47.234220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] MFA Rules not processed for user `b499deffb3164de396174e39db5e932d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:47.235418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:47.258480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:47.300525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1009/2022] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:47.312137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:47.313880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:47.369751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ffc46-b267-4920-8739-aa0431d2c354 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:47.373450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b679f7-bd54-4cf7-bbbe-b2d16a9c4878 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:47.374153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b679f7-bd54-4cf7-bbbe-b2d16a9c4878 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:47.374441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b679f7-bd54-4cf7-bbbe-b2d16a9c4878 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:47.374722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b679f7-bd54-4cf7-bbbe-b2d16a9c4878 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:47.382883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b679f7-bd54-4cf7-bbbe-b2d16a9c4878 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:47.384327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b679f7-bd54-4cf7-bbbe-b2d16a9c4878 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:47.394549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1014/2023] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:57:49.183123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:49.184342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:49.260832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6423d26-cc86-4804-af7f-ba15457db3ae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:49.267783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:49.268702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:49.269079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:49.269514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] PATH_INFO: `/v3/users/c3f02246cfd2475891a6dc1de26a4b95` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:49.270741 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f02246cfd2475891a6dc1de26a4b95)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:49.272678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:49.321452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f02246cfd2475891a6dc1de26a4b95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:49.328428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f02246cfd2475891a6dc1de26a4b95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:49.340021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f02246cfd2475891a6dc1de26a4b95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:49.340676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f02246cfd2475891a6dc1de26a4b95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:49.341312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f02246cfd2475891a6dc1de26a4b95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:49.354266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1010/2024] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:49 2022] DELETE /identity/v3/users/c3f02246cfd2475891a6dc1de26a4b95 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:49.359604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:49.360179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:49.360179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:49.380091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:49.381362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:49.415521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:49.469167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1015/2025] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:49.484916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:49.486396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:49.539624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:49.551495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88610b0-22ca-4e73-a73d-d274bc948d62 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:49.558129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:49.558608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:49.558828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:49.559956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:49.570566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:49.571754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:49.582049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1011/2026] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:49.775671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:49.777381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:49.838572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3000aa0-98fb-465e-809e-d0873c7e8baa None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:49.842738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:49.843397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:49.843681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:49.845409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:49.847415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:49.898227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:49.899647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:49.917575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b4181-6ea7-4dfe-9afb-2f86940ea9d5 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:49.924394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:49.926886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:49.927176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:49.927690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] PATH_INFO: `/v3/projects/6dd701ae977f417dbacafb9e376fb489` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:49.928843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd701ae977f417dbacafb9e376fb489)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:49.930846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:49.938662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:49.941317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:49.967993 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1016/2027] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:50.003649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd701ae977f417dbacafb9e376fb489', 'request_id': 'req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.013827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd701ae977f417dbacafb9e376fb489', 'request_id': 'req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.036920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd701ae977f417dbacafb9e376fb489', 'request_id': 'req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.054525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] The token cache is being invalidate because project 6dd701ae977f417dbacafb9e376fb489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:50.055138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.055900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.060786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1012/2028] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:49 2022] DELETE /identity/v3/projects/6dd701ae977f417dbacafb9e376fb489 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:50.062036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:50.073753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d092f7-b240-4ba0-b276-f4e16a6e9462 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:50.183463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:50.186579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:50.186999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:50.187286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] PATH_INFO: `/v3/users/832532b535b94ce2899df77cbd4554af` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:50.188892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=832532b535b94ce2899df77cbd4554af)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:50.191514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:50.250039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:50.270193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832532b535b94ce2899df77cbd4554af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.322783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832532b535b94ce2899df77cbd4554af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.325838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832532b535b94ce2899df77cbd4554af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.326601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832532b535b94ce2899df77cbd4554af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.343388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-42262ba3-8c50-4077-9827-3f17c5805b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832532b535b94ce2899df77cbd4554af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.372973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1017/2029] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:50 2022] DELETE /identity/v3/users/832532b535b94ce2899df77cbd4554af => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:50.674291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:50.675801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:50.689431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cd4f7-bd70-47fc-ba45-7dcf788304c9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:50.748443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:50.749274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:50.749565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:50.749840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] PATH_INFO: `/v3/projects/295cde2e211040ffa7b50852abf2a17c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:50.751050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=295cde2e211040ffa7b50852abf2a17c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:50.752966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:50.799152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '295cde2e211040ffa7b50852abf2a17c', 'request_id': 'req-c33d13fc-8446-4dc3-860b-a3b769871e4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.807541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '295cde2e211040ffa7b50852abf2a17c', 'request_id': 'req-c33d13fc-8446-4dc3-860b-a3b769871e4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.814502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '295cde2e211040ffa7b50852abf2a17c', 'request_id': 'req-c33d13fc-8446-4dc3-860b-a3b769871e4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.821724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:50.826185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] The token cache is being invalidate because project 295cde2e211040ffa7b50852abf2a17c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:50.826653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.827823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c33d13fc-8446-4dc3-860b-a3b769871e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:50.829590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1013/2030] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:50 2022] DELETE /identity/v3/projects/295cde2e211040ffa7b50852abf2a17c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:50.834473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85366a-516c-4f12-a3fa-b592130b4b7b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:50.834767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85366a-516c-4f12-a3fa-b592130b4b7b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:50.835048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85366a-516c-4f12-a3fa-b592130b4b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:50.853079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ad85366a-516c-4f12-a3fa-b592130b4b7b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:50.854307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ad85366a-516c-4f12-a3fa-b592130b4b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:50.917189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1018/2031] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:50.926436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce9a8e-e711-4a95-b6f0-aca17974e385 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:50.926734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce9a8e-e711-4a95-b6f0-aca17974e385 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:50.927015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce9a8e-e711-4a95-b6f0-aca17974e385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:50.945558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2bce9a8e-e711-4a95-b6f0-aca17974e385 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:50.946799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2bce9a8e-e711-4a95-b6f0-aca17974e385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:50.954853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1014/2032] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:50.959025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85366a-516c-4f12-a3fa-b592130b4b7b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:50.968999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85366a-516c-4f12-a3fa-b592130b4b7b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:50.974038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:50.974694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:50.974981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:50.975259 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:50.975985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:50.976295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:50.977942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:50.991325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1019/2033] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:50.996023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce9a8e-e711-4a95-b6f0-aca17974e385 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.007123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce9a8e-e711-4a95-b6f0-aca17974e385 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.015777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.016883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.017241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.017716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.018567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.020824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.053967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1015/2034] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:50 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:51.055315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.066118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52108d89-321a-44fa-98d6-f37e9514a2e1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.127904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.128597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.128881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.129182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.130291 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.131797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.157216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1020/2035] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:51 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:51.161135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.171804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f8d70-0569-4a14-a44b-8f1ae0f01c12 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.178993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.179666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.179949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.180222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.181237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.182854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.194569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1016/2036] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:57:51.198643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.209645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ebca1-a6e3-407d-b5aa-d51843c45fcb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.214339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb4939-d531-4d3c-98b5-7a4a37ca055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.215015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb4939-d531-4d3c-98b5-7a4a37ca055b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.215350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb4939-d531-4d3c-98b5-7a4a37ca055b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.216216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb4939-d531-4d3c-98b5-7a4a37ca055b admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.216959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb4939-d531-4d3c-98b5-7a4a37ca055b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.218732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb4939-d531-4d3c-98b5-7a4a37ca055b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.229451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1021/2037] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:51.234202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.246170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281b06d-4543-48be-9c6a-101509f7fd96 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.251274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.252077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.252427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.252773 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] PATH_INFO: `/v3/projects/fe2cf38a76bb4d90b98062e143c80df6/users/4978aced742b4ca48e7db88d66192413/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.270436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2cf38a76bb4d90b98062e143c80df6, user_id=4978aced742b4ca48e7db88d66192413, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.273172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.286488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1017/2038] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:51 2022] PUT /identity/v3/projects/fe2cf38a76bb4d90b98062e143c80df6/users/4978aced742b4ca48e7db88d66192413/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:51.296959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85765-e5e1-416a-a932-6db31ddc37a5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.297315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85765-e5e1-416a-a932-6db31ddc37a5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.297632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85765-e5e1-416a-a932-6db31ddc37a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.320988 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:51.324239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-62a85765-e5e1-416a-a932-6db31ddc37a5 None None] MFA Rules not processed for user `4978aced742b4ca48e7db88d66192413`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:51.326074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-62a85765-e5e1-416a-a932-6db31ddc37a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:51.391176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1022/2039] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:51.396092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.457774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40154bda-a6e4-4632-9be9-4a8fba9f12d9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.462635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.463295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.463585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.463861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.464550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.467070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.497040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1018/2040] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:51 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:51.501547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a85765-e5e1-416a-a932-6db31ddc37a5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.512951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a85765-e5e1-416a-a932-6db31ddc37a5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.569750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.570434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.570724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.571007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.571696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.573968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.601405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1023/2041] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:51 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:51.605645 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.616644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecf77c-6732-4aab-b5e8-803e614ce76f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.623016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.623843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.624126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.624409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.625392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.627017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.639537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1019/2042] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:51.643699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.654627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd437970-994b-4ede-aad7-2e9fb71e2844 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.660079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.661030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.661459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.661640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] PATH_INFO: `/v3/projects/7510a3130a3543d195804d4c3513c024/users/9f676a39e12c48d0a06b75b69e9c589c/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.678530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7510a3130a3543d195804d4c3513c024, user_id=9f676a39e12c48d0a06b75b69e9c589c, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.682307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.697456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1024/2043] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:51 2022] PUT /identity/v3/projects/7510a3130a3543d195804d4c3513c024/users/9f676a39e12c48d0a06b75b69e9c589c/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:51.702201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.713364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8979f-d6da-4080-8e15-272fdb5af615 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.788022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3f336-71ba-469d-9c40-71cce8aea279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.788766 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3f336-71ba-469d-9c40-71cce8aea279 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.789056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3f336-71ba-469d-9c40-71cce8aea279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.789364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3f336-71ba-469d-9c40-71cce8aea279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.790094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3f336-71ba-469d-9c40-71cce8aea279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.791683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3f336-71ba-469d-9c40-71cce8aea279 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.802381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1020/2044] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:57:51.813839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.817667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fa7a1-ccf3-4531-835c-9dbf4ea962dc admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.822648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.823533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.823819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.824097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] PATH_INFO: `/v3/projects/7510a3130a3543d195804d4c3513c024/users/9f676a39e12c48d0a06b75b69e9c589c/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.826592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7510a3130a3543d195804d4c3513c024, user_id=9f676a39e12c48d0a06b75b69e9c589c, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.829292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.839575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1025/2045] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:51 2022] PUT /identity/v3/projects/7510a3130a3543d195804d4c3513c024/users/9f676a39e12c48d0a06b75b69e9c589c/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:57:51.850191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d2909-6687-4a97-91f6-42c3c3b657b0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.850482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d2909-6687-4a97-91f6-42c3c3b657b0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.850761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d2909-6687-4a97-91f6-42c3c3b657b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.863127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:51.879708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-199d2909-6687-4a97-91f6-42c3c3b657b0 None None] MFA Rules not processed for user `9f676a39e12c48d0a06b75b69e9c589c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:51.880940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-199d2909-6687-4a97-91f6-42c3c3b657b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:51.949215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1021/2046] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:51.961710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.963497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:52.023939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ba50-20e0-4d7a-8b78-d41f88db2ddf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:52.027762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:52.028407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:52.028684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:52.028953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:52.038455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:52.039819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:52.043639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:52.050768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1026/2047] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:52.364378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d2909-6687-4a97-91f6-42c3c3b657b0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:52.374308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d2909-6687-4a97-91f6-42c3c3b657b0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:52.378975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f676a39e12c48d0a06b75b69e9c589c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7510a3130a3543d195804d4c3513c024', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:52.379644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:52.379935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:52.380216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:52.380946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:52.381291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:52.382888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:52.395856 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1022/2048] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:57:52.401121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:52.410518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eab799-d91e-49a8-8537-3d24164af919 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:52.415689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f676a39e12c48d0a06b75b69e9c589c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7510a3130a3543d195804d4c3513c024', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:52.415925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:52.416174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:52.416449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:52.417132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:52.419588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:52.456913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1027/2049] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:52 2022] POST /identity/v3/projects => generated 354 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:52.460809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:52.476700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18211cba-0016-46e8-b841-267da45138f5 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:52.536735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f676a39e12c48d0a06b75b69e9c589c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7510a3130a3543d195804d4c3513c024', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:52.537384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:52.537670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:52.537946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:52.538704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:52.539126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:52.540600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:52.556949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1023/2050] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:52.593887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:52.605344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d56ec3-d4f1-42a4-b6cc-929139301c14 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:52.610441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f676a39e12c48d0a06b75b69e9c589c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7510a3130a3543d195804d4c3513c024', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:52.611534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:52.611633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:52.611909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] PATH_INFO: `/v3/projects/168e1d00d5074786a9bb37afe08c1a73` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:52.613437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorizing `identity:delete_project(project_id=168e1d00d5074786a9bb37afe08c1a73)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:52.615966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:52.667543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168e1d00d5074786a9bb37afe08c1a73', 'request_id': 'req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:52.677689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168e1d00d5074786a9bb37afe08c1a73', 'request_id': 'req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:52.686773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168e1d00d5074786a9bb37afe08c1a73', 'request_id': 'req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:52.699809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] The token cache is being invalidate because project 168e1d00d5074786a9bb37afe08c1a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:52.700225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:52.700752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b64f8e00-65f7-43b2-a2fe-80d7b4032b5e tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:52.702390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1028/2051] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:52 2022] DELETE /identity/v3/projects/168e1d00d5074786a9bb37afe08c1a73 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:52.900007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:52.902195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:52.969575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f715c1-d000-412d-afed-4f42853ca9c7 tempest-VolumeQuotaClassesTest-943464400 tempest-VolumeQuotaClassesTest-943464400-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:52.988027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:53.049577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:53.050544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:53.050832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:53.051108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] PATH_INFO: `/v3/users/4978aced742b4ca48e7db88d66192413` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:53.052312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4978aced742b4ca48e7db88d66192413)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:53.054300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:53.101194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4978aced742b4ca48e7db88d66192413'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.110948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4978aced742b4ca48e7db88d66192413'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.123414 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4978aced742b4ca48e7db88d66192413'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.124089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4978aced742b4ca48e7db88d66192413'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.124782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4978aced742b4ca48e7db88d66192413'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.136851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1024/2052] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:52 2022] DELETE /identity/v3/users/4978aced742b4ca48e7db88d66192413 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:53.141850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:53.142146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:53.142429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:53.160511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:53.161756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:53.234116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1029/2053] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:53.244409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:53.246048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:53.312078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1212-ffb7-49d6-b6bf-878dfbf8ef18 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:53.315728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:53.316368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:53.316647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:53.316924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:53.325755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:53.327208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:53.336662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1025/2054] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:53.676772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:53.678063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:53.708165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4efe2d-9faa-49cf-baf8-91e0d8af31d9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:53.712871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:53.713790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:53.714043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:53.714321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] PATH_INFO: `/v3/projects/fe2cf38a76bb4d90b98062e143c80df6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:53.716011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2cf38a76bb4d90b98062e143c80df6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:53.717983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:53.796805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2cf38a76bb4d90b98062e143c80df6', 'request_id': 'req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.804606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2cf38a76bb4d90b98062e143c80df6', 'request_id': 'req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.806419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:53.812953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2cf38a76bb4d90b98062e143c80df6', 'request_id': 'req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.831480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] The token cache is being invalidate because project fe2cf38a76bb4d90b98062e143c80df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:53.832000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.832649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.837140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1030/2055] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:53 2022] DELETE /identity/v3/projects/fe2cf38a76bb4d90b98062e143c80df6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:53.838940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:53.850869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399b712-bc68-4627-9ee9-7956df4fc7fd service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:53.907124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:53.926744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:53.927721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:53.927916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:53.928195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] PATH_INFO: `/v3/users/9f676a39e12c48d0a06b75b69e9c589c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:53.929447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f676a39e12c48d0a06b75b69e9c589c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:53.931392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:53.990767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f676a39e12c48d0a06b75b69e9c589c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:53.999117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f676a39e12c48d0a06b75b69e9c589c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.013703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f676a39e12c48d0a06b75b69e9c589c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.014597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f676a39e12c48d0a06b75b69e9c589c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.015209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6f36f32-f38e-4505-8215-655fa0720fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f676a39e12c48d0a06b75b69e9c589c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.030617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1026/2056] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:53 2022] DELETE /identity/v3/users/9f676a39e12c48d0a06b75b69e9c589c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:54.369048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:54.370705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:54.455298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83dfd0-7cfd-4b17-9318-028d024f9ca8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:54.511528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:54.513236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:54.513578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:54.513870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] PATH_INFO: `/v3/projects/7510a3130a3543d195804d4c3513c024` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:54.515154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7510a3130a3543d195804d4c3513c024)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:54.517095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:54.564836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7510a3130a3543d195804d4c3513c024', 'request_id': 'req-1cc446ba-7eb1-4087-af4b-29e885b00997'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.570892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7510a3130a3543d195804d4c3513c024', 'request_id': 'req-1cc446ba-7eb1-4087-af4b-29e885b00997'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.578252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7510a3130a3543d195804d4c3513c024', 'request_id': 'req-1cc446ba-7eb1-4087-af4b-29e885b00997'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.593827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] The token cache is being invalidate because project 7510a3130a3543d195804d4c3513c024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:54.594125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.594986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1cc446ba-7eb1-4087-af4b-29e885b00997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:54.596597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1031/2057] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:54 2022] DELETE /identity/v3/projects/7510a3130a3543d195804d4c3513c024 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:54.601583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34562d-150a-4c6c-bc84-b0234de37706 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:54.601875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34562d-150a-4c6c-bc84-b0234de37706 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:54.602154 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34562d-150a-4c6c-bc84-b0234de37706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:54.649850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dd34562d-150a-4c6c-bc84-b0234de37706 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:54.651068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd34562d-150a-4c6c-bc84-b0234de37706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:54.712641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1027/2058] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:54.721631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cde52-8df7-42fd-ad60-e44e27b19288 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:54.722125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cde52-8df7-42fd-ad60-e44e27b19288 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:54.722226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cde52-8df7-42fd-ad60-e44e27b19288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:54.740858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6a3cde52-8df7-42fd-ad60-e44e27b19288 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:54.742069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6a3cde52-8df7-42fd-ad60-e44e27b19288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:54.750405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1032/2059] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:54.754588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34562d-150a-4c6c-bc84-b0234de37706 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:54.764185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34562d-150a-4c6c-bc84-b0234de37706 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:54.768958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:54.769652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:54.769939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:54.770244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:54.770980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:54.771311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:54.772939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:54.785168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1028/2060] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:54.789367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cde52-8df7-42fd-ad60-e44e27b19288 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:54.795418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:54.799067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cde52-8df7-42fd-ad60-e44e27b19288 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:54.804692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:54.805372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:54.805844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:54.806260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:54.807317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:54.810339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:54.850159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1033/2061] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:54 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:54.854247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:54.864679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa35e5-8c81-4214-b712-4323252298b7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:54.889665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:54.929867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:54.930674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:54.930963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:54.931275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:54.932012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:54.934712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:54.957461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1029/2062] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:54 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:54.961775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:54.974085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c349e23-50eb-4cb5-9d3f-5f8312c4e5c6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:54.978856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:54.979523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:54.979807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:54.980085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:54.980813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:54.983000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:54.993842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1034/2063] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:54.998227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:55.009075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6dfb0-4513-4aa8-8388-724e63774f52 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:55.013875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154ddfe-4829-43ce-9e72-11eecbe40eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:55.019971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154ddfe-4829-43ce-9e72-11eecbe40eca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:55.019971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154ddfe-4829-43ce-9e72-11eecbe40eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:55.019971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154ddfe-4829-43ce-9e72-11eecbe40eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:55.019971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154ddfe-4829-43ce-9e72-11eecbe40eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:55.019971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154ddfe-4829-43ce-9e72-11eecbe40eca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:55.032207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1030/2064] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:55.033486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:55.043451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b54d8-6e2a-43f9-8b67-465df7ecfded admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:55.049422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:55.050311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:55.050592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:55.050869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] PATH_INFO: `/v3/projects/a725b19ff51c4cc9ac71f8a8cced6b1b/users/784625a5936f44359533d9476e7f9061/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:55.068457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a725b19ff51c4cc9ac71f8a8cced6b1b, user_id=784625a5936f44359533d9476e7f9061, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:55.071284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:55.086055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1035/2065] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:55 2022] PUT /identity/v3/projects/a725b19ff51c4cc9ac71f8a8cced6b1b/users/784625a5936f44359533d9476e7f9061/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:55.097601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:55.097990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:55.098374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:55.126861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] MFA Rules not processed for user `784625a5936f44359533d9476e7f9061`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:55.128088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:55.218325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1031/2066] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:55.229432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:55.231134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:55.307530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2ee9-1e0b-478b-bdbf-85d805d53e44 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:55.311389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4df72d-32ac-4ee0-9843-8a4cb7e7f5f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:55.312224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4df72d-32ac-4ee0-9843-8a4cb7e7f5f9 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:55.312612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4df72d-32ac-4ee0-9843-8a4cb7e7f5f9 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:55.313009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4df72d-32ac-4ee0-9843-8a4cb7e7f5f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:55.328317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4df72d-32ac-4ee0-9843-8a4cb7e7f5f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:55.329840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4df72d-32ac-4ee0-9843-8a4cb7e7f5f9 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:55.341898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1036/2067] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:56.049306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:56.050717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:56.117715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dfca3-bdde-40a8-ad77-6ca866b8ecc4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:56.123105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:56.124008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:56.124286 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:56.124554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] PATH_INFO: `/v3/users/6fe2c541608f404298577c8090d4f69f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:56.125771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe2c541608f404298577c8090d4f69f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:56.127738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:56.165650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:56.176774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe2c541608f404298577c8090d4f69f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.185598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe2c541608f404298577c8090d4f69f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.199664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe2c541608f404298577c8090d4f69f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.200516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe2c541608f404298577c8090d4f69f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.201079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe2c541608f404298577c8090d4f69f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.213086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1032/2068] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:56 2022] DELETE /identity/v3/users/6fe2c541608f404298577c8090d4f69f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:56.217611 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:56.218075 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:56.218168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:56.236061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:56.237273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:56.303415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1037/2069] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:56.316206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:56.317654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:56.433844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c4eb7-ab70-4344-a2bc-5496a6f4d720 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:56.437917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:56.438572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:56.438850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:56.439121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:56.449233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:56.450695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:56.460508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1033/2070] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:57:56.772255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:56.773818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:56.787492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608239e6-5e18-4fa7-83a7-a219191ac291 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:56.793419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:56.796147 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:56.796147 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:56.796147 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] PATH_INFO: `/v3/projects/70e1d74aa6364176aced7e3bcdc078c1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:56.797094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e1d74aa6364176aced7e3bcdc078c1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:56.799528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:56.850372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:56.877254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e1d74aa6364176aced7e3bcdc078c1', 'request_id': 'req-1e6e5d41-bbde-43c1-adef-82d34698a2f6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.883614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e1d74aa6364176aced7e3bcdc078c1', 'request_id': 'req-1e6e5d41-bbde-43c1-adef-82d34698a2f6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.890058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e1d74aa6364176aced7e3bcdc078c1', 'request_id': 'req-1e6e5d41-bbde-43c1-adef-82d34698a2f6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.903765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] The token cache is being invalidate because project 70e1d74aa6364176aced7e3bcdc078c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:56.904135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.904817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:56.906544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1038/2071] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:56 2022] DELETE /identity/v3/projects/70e1d74aa6364176aced7e3bcdc078c1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:56.910406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:56.923183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df698c61-a86d-4bfa-b19c-bc7aebbe3773 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:56.984051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:56.984966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:56.985310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:56.985593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] PATH_INFO: `/v3/users/eaf07abffe9243a49d9948e929659c33` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:56.986994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf07abffe9243a49d9948e929659c33)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:56.989348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:57.053000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:57.061285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf07abffe9243a49d9948e929659c33'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.070133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf07abffe9243a49d9948e929659c33'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.086233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf07abffe9243a49d9948e929659c33'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.087077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf07abffe9243a49d9948e929659c33'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.087676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-085b26ef-dcb0-46c0-b23e-4b6f02a0acac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf07abffe9243a49d9948e929659c33'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.100927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1034/2072] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:57:56 2022] DELETE /identity/v3/users/eaf07abffe9243a49d9948e929659c33 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:57.395104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:57.396953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:57.414581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5d41-bbde-43c1-adef-82d34698a2f6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:57.492445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:57.493346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:57.493636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:57.493913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] PATH_INFO: `/v3/projects/db8902bb66da4ebe9aa7a7f9832b7aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:57.495113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8902bb66da4ebe9aa7a7f9832b7aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:57.497854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:57.552574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8902bb66da4ebe9aa7a7f9832b7aac', 'request_id': 'req-2bc11b6b-4db8-4998-ad0b-144c29084e7a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.559155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8902bb66da4ebe9aa7a7f9832b7aac', 'request_id': 'req-2bc11b6b-4db8-4998-ad0b-144c29084e7a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.566677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8902bb66da4ebe9aa7a7f9832b7aac', 'request_id': 'req-2bc11b6b-4db8-4998-ad0b-144c29084e7a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.585756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] The token cache is being invalidate because project db8902bb66da4ebe9aa7a7f9832b7aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:57:57.586202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.586945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2bc11b6b-4db8-4998-ad0b-144c29084e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:57:57.588570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1039/2073] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:57:57 2022] DELETE /identity/v3/projects/db8902bb66da4ebe9aa7a7f9832b7aac => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:57.595283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5b6d8-2172-45ea-9e54-77c52c82b098 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:57.595818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5b6d8-2172-45ea-9e54-77c52c82b098 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:57.596129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5b6d8-2172-45ea-9e54-77c52c82b098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:57.617656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-66a5b6d8-2172-45ea-9e54-77c52c82b098 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:57.619003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-66a5b6d8-2172-45ea-9e54-77c52c82b098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:57.684372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1035/2074] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:57.692608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d239b9-560d-4dbf-b397-3fa91ef14a46 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:57.692896 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d239b9-560d-4dbf-b397-3fa91ef14a46 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:57.693201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d239b9-560d-4dbf-b397-3fa91ef14a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:57.713107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-61d239b9-560d-4dbf-b397-3fa91ef14a46 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:57.714369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-61d239b9-560d-4dbf-b397-3fa91ef14a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:57.722936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1040/2075] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:57:57.727172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5b6d8-2172-45ea-9e54-77c52c82b098 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:57.736807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5b6d8-2172-45ea-9e54-77c52c82b098 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:57.741399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:57.742057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:57.742946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:57.742946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:57.743394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:57:57.743708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:57.745365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:57.765050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1036/2076] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:57:57.766186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d239b9-560d-4dbf-b397-3fa91ef14a46 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:57.775428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d239b9-560d-4dbf-b397-3fa91ef14a46 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:57.780984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:57.781688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:57.781978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:57.782257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:57.782944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:57.784889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:57.813711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1041/2077] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:57:57 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:57:57.815986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:57.828193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2b667-5740-43c5-b30d-4150ed78677c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:57.937201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:57.938235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:57.938584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:57.938929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:57.939726 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:57.941791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:57.976168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1037/2078] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:57:57 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:57:57.980335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:57.990886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94148157-c9a6-4664-8394-c1890f0af6b5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:57.997914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:57.998684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:57.999040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:57.999389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:58.000199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:58.001938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:58.017126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1042/2079] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:58.018618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:58.029840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fb60a-45aa-4619-b6d7-7557436018d4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:58.034529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b6fb9-cb0e-4b75-b08e-200ee9c3668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:58.035202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b6fb9-cb0e-4b75-b08e-200ee9c3668d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:58.035482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b6fb9-cb0e-4b75-b08e-200ee9c3668d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:58.035754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b6fb9-cb0e-4b75-b08e-200ee9c3668d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:58.036497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b6fb9-cb0e-4b75-b08e-200ee9c3668d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:58.038303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b6fb9-cb0e-4b75-b08e-200ee9c3668d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:58.049649 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1038/2080] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:57:58.053743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:58.067169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51f2da-0458-4c65-b528-b7b7559a63ad admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:58.079728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:58.080934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:58.081596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:58.082199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] PATH_INFO: `/v3/projects/0e0fb36c1dc945419b201c725a0c7bbd/users/91c1d39e9f7e4b11bfe56b967dc3b5d2/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:58.101683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:58.123290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0fb36c1dc945419b201c725a0c7bbd, user_id=91c1d39e9f7e4b11bfe56b967dc3b5d2, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:58.126585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:58.147451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1043/2081] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:57:58 2022] PUT /identity/v3/projects/0e0fb36c1dc945419b201c725a0c7bbd/users/91c1d39e9f7e4b11bfe56b967dc3b5d2/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:57:58.165721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:58.166751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:58.167187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:58.220257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] MFA Rules not processed for user `91c1d39e9f7e4b11bfe56b967dc3b5d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:57:58.221339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:58.232282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:58.332883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1039/2082] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:57:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:57:58.342289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:58.345771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:58.442238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27253d1f-13ba-47ec-acd8-273920f758d3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:58.447808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:58.448653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:58.449194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:58.450162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:58.460540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:58.462397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:58.474763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1044/2083] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:57:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:03.641759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:03.643405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:03.728672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee168d-e37d-4dda-add6-49b61bcd6075 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:03.733933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:03.734814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:03.735142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:03.735406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] PATH_INFO: `/v3/users/bda029aace2c4e40bcc470138243d0da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:03.736695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda029aace2c4e40bcc470138243d0da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:03.738633 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:03.788175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda029aace2c4e40bcc470138243d0da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:03.796069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda029aace2c4e40bcc470138243d0da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:03.807877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda029aace2c4e40bcc470138243d0da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:03.808514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda029aace2c4e40bcc470138243d0da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:03.809113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53907d01-b297-4b94-923e-8586e49c3be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda029aace2c4e40bcc470138243d0da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:03.822059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1040/2084] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:03 2022] DELETE /identity/v3/users/bda029aace2c4e40bcc470138243d0da => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:04.004855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.017300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af40e72-2f80-4b9b-b301-09f9d538fa55 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.092809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.093945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.094473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.094759 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] PATH_INFO: `/v3/projects/9ed49180e7f24cb296141d06576ead81` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.096133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed49180e7f24cb296141d06576ead81)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.098193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.147777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed49180e7f24cb296141d06576ead81', 'request_id': 'req-c01a71aa-b60b-43c3-816b-b1abc27df08e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:04.154308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed49180e7f24cb296141d06576ead81', 'request_id': 'req-c01a71aa-b60b-43c3-816b-b1abc27df08e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:04.159935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed49180e7f24cb296141d06576ead81', 'request_id': 'req-c01a71aa-b60b-43c3-816b-b1abc27df08e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:04.172315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] The token cache is being invalidate because project 9ed49180e7f24cb296141d06576ead81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:04.172602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:04.173260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c01a71aa-b60b-43c3-816b-b1abc27df08e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:04.174878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1045/2085] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:04 2022] DELETE /identity/v3/projects/9ed49180e7f24cb296141d06576ead81 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:04.180056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e6a4-9dce-467d-9db9-b95b6cd23605 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.180377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e6a4-9dce-467d-9db9-b95b6cd23605 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.180689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413e6a4-9dce-467d-9db9-b95b6cd23605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.206556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5413e6a4-9dce-467d-9db9-b95b6cd23605 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:04.207777 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5413e6a4-9dce-467d-9db9-b95b6cd23605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:04.235291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:04.280926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1041/2086] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:04.288956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab9f8-59eb-471c-9af6-e870926659dc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.289279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab9f8-59eb-471c-9af6-e870926659dc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.289796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab9f8-59eb-471c-9af6-e870926659dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.308338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-33aab9f8-59eb-471c-9af6-e870926659dc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:04.309589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33aab9f8-59eb-471c-9af6-e870926659dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:04.320621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1046/2087] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:04.324844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413e6a4-9dce-467d-9db9-b95b6cd23605 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.334973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413e6a4-9dce-467d-9db9-b95b6cd23605 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.339693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.340356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.340665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.340935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.341752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:58:04.342065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.343666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.360104 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1042/2088] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:58:04.361960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab9f8-59eb-471c-9af6-e870926659dc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.371779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab9f8-59eb-471c-9af6-e870926659dc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.377227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.377900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.378186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.378467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.379227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.381188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.389778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:04.412791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1047/2089] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:04 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:04.414211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.430198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8f80a-3837-431c-b899-6fd452ec0667 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.485827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.486500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.486784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.487063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.487760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.489694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.514970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1043/2090] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:04 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:04.518860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.534950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92288688-4504-44c4-b706-82b2f784de8d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.539895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.540596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.540883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.541187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.541923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.543549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.554792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1048/2091] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:58:04.558601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.569180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab26d7-6c00-403b-9d7c-92fe07fade82 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.573820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22338d-d924-476f-9841-dc13079ae5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.574486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22338d-d924-476f-9841-dc13079ae5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.574775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22338d-d924-476f-9841-dc13079ae5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.575057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22338d-d924-476f-9841-dc13079ae5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.575786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22338d-d924-476f-9841-dc13079ae5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.577506 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22338d-d924-476f-9841-dc13079ae5c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.588592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1044/2092] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:04.593346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.603428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b2e4-393d-4fac-ad1a-8a4d237aa1fd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.610253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.611154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.611446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.611723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] PATH_INFO: `/v3/projects/161b3525d8964113888f5f859dac04d2/users/07c6657dedee4e9b9a6a8dc40ecdbbd1/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.635946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=161b3525d8964113888f5f859dac04d2, user_id=07c6657dedee4e9b9a6a8dc40ecdbbd1, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.639552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.653749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1049/2093] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:04 2022] PUT /identity/v3/projects/161b3525d8964113888f5f859dac04d2/users/07c6657dedee4e9b9a6a8dc40ecdbbd1/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:04.665271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad30b7-ae03-4275-9ffd-c2b729088244 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.665785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad30b7-ae03-4275-9ffd-c2b729088244 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.665888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad30b7-ae03-4275-9ffd-c2b729088244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.695096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a7ad30b7-ae03-4275-9ffd-c2b729088244 None None] MFA Rules not processed for user `07c6657dedee4e9b9a6a8dc40ecdbbd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:04.696469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7ad30b7-ae03-4275-9ffd-c2b729088244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:04.768216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1045/2094] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:04.772713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.835306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b262c88-1555-457f-adb8-3acd95d9e306 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.840173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.840833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.841114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.841419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.842642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.844815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.876314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1050/2095] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:04 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:04.877473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad30b7-ae03-4275-9ffd-c2b729088244 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:04.897519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad30b7-ae03-4275-9ffd-c2b729088244 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:04.928159 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:04.963639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:04.964444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:04.964729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:04.965246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:04.966046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:04.969601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:04.998538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1046/2096] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:04 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:04.999779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:05.011438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cb378-ed88-4c10-bfc6-fd237d9f590d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:05.017373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:05.018402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:05.018508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:05.018820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:05.019668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:05.021659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:05.036364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1051/2097] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:05.041118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:05.051996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875347f-6550-4a10-9d62-5339af5c5d54 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:05.058360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:05.059415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:05.059730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:05.060012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] PATH_INFO: `/v3/projects/cf10951e4c424f85b2baea31db29ee7d/users/7cba2e5a5d0744f88cc7b526e0bc74be/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:05.076558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf10951e4c424f85b2baea31db29ee7d, user_id=7cba2e5a5d0744f88cc7b526e0bc74be, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:05.079268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:05.094949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1047/2098] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:05 2022] PUT /identity/v3/projects/cf10951e4c424f85b2baea31db29ee7d/users/7cba2e5a5d0744f88cc7b526e0bc74be/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:05.099147 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:05.111305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35565214-958b-497e-b1df-b586bc287fc6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:05.173902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d82237-ea28-4b9c-b0d3-64428ede8c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:05.174573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82237-ea28-4b9c-b0d3-64428ede8c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:05.174858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82237-ea28-4b9c-b0d3-64428ede8c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:05.175137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82237-ea28-4b9c-b0d3-64428ede8c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:05.175864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d82237-ea28-4b9c-b0d3-64428ede8c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:05.177498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d82237-ea28-4b9c-b0d3-64428ede8c12 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:05.187642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1052/2099] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:05.191787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:05.202752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042a2f1-a80f-4697-ac7e-1d32332f6478 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:05.207410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:05.208312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:05.208594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:05.208872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] PATH_INFO: `/v3/projects/cf10951e4c424f85b2baea31db29ee7d/users/7cba2e5a5d0744f88cc7b526e0bc74be/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:05.210866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf10951e4c424f85b2baea31db29ee7d, user_id=7cba2e5a5d0744f88cc7b526e0bc74be, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:05.213529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:05.224361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1048/2100] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:05 2022] PUT /identity/v3/projects/cf10951e4c424f85b2baea31db29ee7d/users/7cba2e5a5d0744f88cc7b526e0bc74be/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:05.234387 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae6d2a-aca7-4019-963d-b032479565b8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:05.234704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae6d2a-aca7-4019-963d-b032479565b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:05.235013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae6d2a-aca7-4019-963d-b032479565b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:05.261673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e3ae6d2a-aca7-4019-963d-b032479565b8 None None] MFA Rules not processed for user `7cba2e5a5d0744f88cc7b526e0bc74be`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:05.262886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e3ae6d2a-aca7-4019-963d-b032479565b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:05.274944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:05.348200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1053/2101] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:05 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:05.359398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:05.361125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:05.423082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ea1a8-a169-4bba-947b-f3e928428024 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:05.426813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941722f9-b750-40d9-9766-8a0009bd2cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:05.427456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941722f9-b750-40d9-9766-8a0009bd2cee service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:05.427742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941722f9-b750-40d9-9766-8a0009bd2cee service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:05.428019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941722f9-b750-40d9-9766-8a0009bd2cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:05.438580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941722f9-b750-40d9-9766-8a0009bd2cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:05.440072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941722f9-b750-40d9-9766-8a0009bd2cee service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:05.450070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1049/2102] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:05.860205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae6d2a-aca7-4019-963d-b032479565b8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:05.930023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae6d2a-aca7-4019-963d-b032479565b8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:05.934435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:05.935304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:05.936149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:05.936424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] PATH_INFO: `/v3/users/07c6657dedee4e9b9a6a8dc40ecdbbd1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:05.937633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c6657dedee4e9b9a6a8dc40ecdbbd1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:05.939581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:05.986386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c6657dedee4e9b9a6a8dc40ecdbbd1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:05.999399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c6657dedee4e9b9a6a8dc40ecdbbd1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.000110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c6657dedee4e9b9a6a8dc40ecdbbd1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.000709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c6657dedee4e9b9a6a8dc40ecdbbd1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.008187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c6657dedee4e9b9a6a8dc40ecdbbd1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.021678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1054/2103] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:05 2022] DELETE /identity/v3/users/07c6657dedee4e9b9a6a8dc40ecdbbd1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:06.027031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:06.027756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:06.027756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:06.046890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:06.048019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:06.109750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1050/2104] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:06.120139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:06.122198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:06.155483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:06.184893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183f09b-0c27-4db2-a3c6-25ee0c57eb86 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:06.189226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:06.189918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:06.190206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:06.190489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:06.200186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:06.201956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:06.212383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1055/2105] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:06.494288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:06.495920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:06.508005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec5369-2102-417f-9ab2-77e63fd76df4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:06.513967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:06.514902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:06.515254 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:06.515582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] PATH_INFO: `/v3/projects/161b3525d8964113888f5f859dac04d2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:06.516843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=161b3525d8964113888f5f859dac04d2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:06.518793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:06.537457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:06.567941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '161b3525d8964113888f5f859dac04d2', 'request_id': 'req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.577349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '161b3525d8964113888f5f859dac04d2', 'request_id': 'req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.583793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '161b3525d8964113888f5f859dac04d2', 'request_id': 'req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.595308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] The token cache is being invalidate because project 161b3525d8964113888f5f859dac04d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:06.595607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.596249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.597838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1051/2106] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:06 2022] DELETE /identity/v3/projects/161b3525d8964113888f5f859dac04d2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:06.601801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:06.611510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c230f0-953f-4d85-b83e-2b96b9ea0323 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:06.672986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:06.673996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:06.674289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:06.674566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] PATH_INFO: `/v3/users/7cba2e5a5d0744f88cc7b526e0bc74be` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:06.675787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cba2e5a5d0744f88cc7b526e0bc74be)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:06.677879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:06.729557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cba2e5a5d0744f88cc7b526e0bc74be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.742300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cba2e5a5d0744f88cc7b526e0bc74be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.743011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cba2e5a5d0744f88cc7b526e0bc74be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.743757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cba2e5a5d0744f88cc7b526e0bc74be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.751695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3fa8cd1f-1ba2-49ad-aa41-f8529b48ea05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cba2e5a5d0744f88cc7b526e0bc74be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:06.765532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1056/2107] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:06 2022] DELETE /identity/v3/users/7cba2e5a5d0744f88cc7b526e0bc74be => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:07.033266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.034911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:07.056839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f1938-2a55-4294-a3aa-cfc2e0f33062 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.124844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.125755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.126057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.126315 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] PATH_INFO: `/v3/projects/cf10951e4c424f85b2baea31db29ee7d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.127572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf10951e4c424f85b2baea31db29ee7d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.129549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.183972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf10951e4c424f85b2baea31db29ee7d', 'request_id': 'req-fbebc193-2f24-4574-8c94-9a5ca97c0886'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:07.204526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf10951e4c424f85b2baea31db29ee7d', 'request_id': 'req-fbebc193-2f24-4574-8c94-9a5ca97c0886'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:07.212071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf10951e4c424f85b2baea31db29ee7d', 'request_id': 'req-fbebc193-2f24-4574-8c94-9a5ca97c0886'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:07.223749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] The token cache is being invalidate because project cf10951e4c424f85b2baea31db29ee7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:07.224093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:07.224772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fbebc193-2f24-4574-8c94-9a5ca97c0886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:07.226626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1052/2108] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:07 2022] DELETE /identity/v3/projects/cf10951e4c424f85b2baea31db29ee7d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:07.232812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9dcd5-d3c1-4d2d-b3bf-f7d611e5679f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.233383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9dcd5-d3c1-4d2d-b3bf-f7d611e5679f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.233383 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9dcd5-d3c1-4d2d-b3bf-f7d611e5679f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.251893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e5f9dcd5-d3c1-4d2d-b3bf-f7d611e5679f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:07.253106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e5f9dcd5-d3c1-4d2d-b3bf-f7d611e5679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:07.306972 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:07.326137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1057/2109] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:07.334109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c39f-35b1-475b-8a42-626c45813e07 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.334425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c39f-35b1-475b-8a42-626c45813e07 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.334709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4c39f-35b1-475b-8a42-626c45813e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.354705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1ee4c39f-35b1-475b-8a42-626c45813e07 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:07.355966 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ee4c39f-35b1-475b-8a42-626c45813e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:07.364485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1053/2110] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:07.369033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9dcd5-d3c1-4d2d-b3bf-f7d611e5679f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.379313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9dcd5-d3c1-4d2d-b3bf-f7d611e5679f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.384097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.384886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.385120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.385468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.386296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:58:07.386645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.388845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.402958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1058/2111] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:58:07.408054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4c39f-35b1-475b-8a42-626c45813e07 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.417740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4c39f-35b1-475b-8a42-626c45813e07 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.423672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.424394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.424655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.424933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.425693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.427911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.477249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1054/2112] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:07 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:07.481315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.494132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33fc3e-614c-4530-a8da-0462b1641f7f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.561790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.562471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.562761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.563042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.563747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.565708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.585733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1059/2113] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:07 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:07.590059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.600752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567891d4-6dac-4b9e-852e-cbf747eca02d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.605451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.606143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.606428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.606703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.608098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.609941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.611348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:07.623631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1055/2114] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:07.627198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.637028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dcd01-1316-4f5c-84ec-1750068b0cd6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.641731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845b2a7-441c-436b-bcb5-bf1356fefe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.642395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845b2a7-441c-436b-bcb5-bf1356fefe57 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.642681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845b2a7-441c-436b-bcb5-bf1356fefe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.642960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845b2a7-441c-436b-bcb5-bf1356fefe57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.643687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845b2a7-441c-436b-bcb5-bf1356fefe57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.645314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845b2a7-441c-436b-bcb5-bf1356fefe57 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.655971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1060/2115] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:07.659892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.670711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d4d1f-1932-4206-b366-ef4c68c763d5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.675673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.676560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.676927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.677680 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] PATH_INFO: `/v3/projects/fa35fd7465ae472184c6c05bb069c2fd/users/9276b4d8fdf6484c80875bf4d151cb00/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.696425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa35fd7465ae472184c6c05bb069c2fd, user_id=9276b4d8fdf6484c80875bf4d151cb00, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.699121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.711859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1056/2116] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:07 2022] PUT /identity/v3/projects/fa35fd7465ae472184c6c05bb069c2fd/users/9276b4d8fdf6484c80875bf4d151cb00/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:07.723618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9b03a-b128-490e-9a43-cc8d005b3ca8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.723912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9b03a-b128-490e-9a43-cc8d005b3ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.724216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9b03a-b128-490e-9a43-cc8d005b3ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.751366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d3f9b03a-b128-490e-9a43-cc8d005b3ca8 None None] MFA Rules not processed for user `9276b4d8fdf6484c80875bf4d151cb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:07.752571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d3f9b03a-b128-490e-9a43-cc8d005b3ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:07.818412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1061/2117] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:07.822609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.887546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37ada7-dc53-4a29-a70c-455344a749b5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.893932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:07.894607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:07.894915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:07.895187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:07.895885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:07.898058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:07.933184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1057/2118] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:07 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:07.937623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9b03a-b128-490e-9a43-cc8d005b3ca8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:07.948525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9b03a-b128-490e-9a43-cc8d005b3ca8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:07.994703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:08.008579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.009207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.009498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.009779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.010466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.012440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.037344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1062/2119] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:07 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:08.038595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.049118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750528ff-f40d-4f96-ae6f-43500164c44a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.053830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.054483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.054769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.055044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.055767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.057401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.068010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1058/2120] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:08.072067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.082433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6214-07b7-4631-9014-67a77fe19451 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.087054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6df81-3083-49bc-a648-0fa888ff6bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.087711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6df81-3083-49bc-a648-0fa888ff6bff admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.087994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6df81-3083-49bc-a648-0fa888ff6bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.088269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6df81-3083-49bc-a648-0fa888ff6bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.088991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6df81-3083-49bc-a648-0fa888ff6bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.090608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6df81-3083-49bc-a648-0fa888ff6bff admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.101636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1063/2121] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:08.106497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.117049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31f26b-863d-44c3-9ac7-568dfccdb9ca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.121929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.122846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.123134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.123414 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] PATH_INFO: `/v3/projects/c9e89baac8414599a12858ba60ac9fd7/users/82e38f926d5a437ea35bbeb2a680c702/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.137945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e89baac8414599a12858ba60ac9fd7, user_id=82e38f926d5a437ea35bbeb2a680c702, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.141308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.154925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1059/2122] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:08 2022] PUT /identity/v3/projects/c9e89baac8414599a12858ba60ac9fd7/users/82e38f926d5a437ea35bbeb2a680c702/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:58:08.165097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fae3e6-884b-4bf4-8fb2-02aad6b772e5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.165433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fae3e6-884b-4bf4-8fb2-02aad6b772e5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.165706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fae3e6-884b-4bf4-8fb2-02aad6b772e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.191985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-86fae3e6-884b-4bf4-8fb2-02aad6b772e5 None None] MFA Rules not processed for user `82e38f926d5a437ea35bbeb2a680c702`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:08.193325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-86fae3e6-884b-4bf4-8fb2-02aad6b772e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:08.257074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1064/2123] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:08.261422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.335566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7586f2-c750-4846-9610-908ec5961911 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.340347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.341007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.341323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.342266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.343011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.344912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.370129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1060/2124] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:08 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:08.374842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fae3e6-884b-4bf4-8fb2-02aad6b772e5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.386988 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fae3e6-884b-4bf4-8fb2-02aad6b772e5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.449569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01df2b-d994-46b6-b31d-bf800fcba66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.450252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01df2b-d994-46b6-b31d-bf800fcba66e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.450541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01df2b-d994-46b6-b31d-bf800fcba66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.450818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01df2b-d994-46b6-b31d-bf800fcba66e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.452140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01df2b-d994-46b6-b31d-bf800fcba66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.454118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01df2b-d994-46b6-b31d-bf800fcba66e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.479028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1065/2125] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:08 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:08.487725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.508073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d821517-7322-46d5-8c8b-729a55ca8add admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.509890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.510189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.510472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.514668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.515699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.516359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.516359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.516996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.518621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.520447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:08.531424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1061/2126] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:08.535565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.545090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:08.546338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:08.546704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74dfd3-58c1-4127-996b-c9b756a9d1c9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.551615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.552743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.553016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.553411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] PATH_INFO: `/v3/projects/ad8019ed81224ed088e2ed4ea724a994/users/3684137ae5d04b7e9e68d7a4c74b2036/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.573182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8019ed81224ed088e2ed4ea724a994, user_id=3684137ae5d04b7e9e68d7a4c74b2036, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.579014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.593231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:08.595024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1062/2127] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:08 2022] PUT /identity/v3/projects/ad8019ed81224ed088e2ed4ea724a994/users/3684137ae5d04b7e9e68d7a4c74b2036/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:58:08.598983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.620583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1066/2128] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:58:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:08.663976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4cbc8-b4f8-4a83-8882-f1ee2c75d640 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.664830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.666363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:08.668747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.669455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.669772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.670047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.670775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.672851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.682532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1063/2129] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:58:08.687126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.696684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04233cf-3b83-4442-a6f4-1115aea3f912 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.702018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb1ce4-c164-4739-9e23-8b1d764a5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.702911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb1ce4-c164-4739-9e23-8b1d764a5b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.703188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb1ce4-c164-4739-9e23-8b1d764a5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.703464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb1ce4-c164-4739-9e23-8b1d764a5b6d admin admin] PATH_INFO: `/v3/projects/ad8019ed81224ed088e2ed4ea724a994/users/3684137ae5d04b7e9e68d7a4c74b2036/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.705594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb1ce4-c164-4739-9e23-8b1d764a5b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8019ed81224ed088e2ed4ea724a994, user_id=3684137ae5d04b7e9e68d7a4c74b2036, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.708396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb1ce4-c164-4739-9e23-8b1d764a5b6d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.720592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1064/2130] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:08 2022] PUT /identity/v3/projects/ad8019ed81224ed088e2ed4ea724a994/users/3684137ae5d04b7e9e68d7a4c74b2036/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:58:08.730752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.731129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.731486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.744137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31388ada-f300-4e74-9d2e-7aec485ac55d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.759695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] MFA Rules not processed for user `3684137ae5d04b7e9e68d7a4c74b2036`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:08.760905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:08.804300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c37fd-12c4-47ab-b529-029b23f7deea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.804971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c37fd-12c4-47ab-b529-029b23f7deea service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.805296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c37fd-12c4-47ab-b529-029b23f7deea service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.805692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c37fd-12c4-47ab-b529-029b23f7deea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.830399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1065/2131] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:08.841798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:08.843148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:08.862729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c37fd-12c4-47ab-b529-029b23f7deea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.864330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c37fd-12c4-47ab-b529-029b23f7deea service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.872431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1067/2132] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 13:58:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 13:58:08.888123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:08.904411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32396f87-2628-40ed-a012-9f6bec6ed776 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:08.908738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:08.909427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:08.909715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:08.910000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:08.920912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:08.922387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:08.932171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1066/2133] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:58:09.450614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:09.451104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:09.451462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:09.469306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:09.470586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:09.477700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1068/2134] 10.222.0.91 () {58 vars in 967 bytes} [Thu May 26 13:58:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:09.490716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:09.499479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:09.500911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:09.502384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5aba-f112-4a8b-b6db-c3c0303d0b4d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:09.507348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250c7e6-d7e0-4736-b818-7370d1f0aadb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:09.508066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250c7e6-d7e0-4736-b818-7370d1f0aadb service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:09.508448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250c7e6-d7e0-4736-b818-7370d1f0aadb service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:09.508796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250c7e6-d7e0-4736-b818-7370d1f0aadb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:09.514072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26cf8-f2a9-4924-aa10-02bf3c8d99d5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:09.516945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:09.517728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:09.518087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:09.518440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:09.518541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250c7e6-d7e0-4736-b818-7370d1f0aadb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:09.520208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250c7e6-d7e0-4736-b818-7370d1f0aadb service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:09.529803 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1067/2135] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 13:58:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:58:09.536528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:09.593401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:09.595457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:09.606785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1069/2136] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:58:09.743431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdaa7b-d124-4e65-901b-b93a5b644da9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:09.743713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdaa7b-d124-4e65-901b-b93a5b644da9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:09.744020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdaa7b-d124-4e65-901b-b93a5b644da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:09.762271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2afdaa7b-d124-4e65-901b-b93a5b644da9 None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:09.764105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2afdaa7b-d124-4e65-901b-b93a5b644da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:09.772119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1068/2137] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:58:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:09.785929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:09.797627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4acc9-2323-401d-b07e-052d5d5240d3 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:09.803044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:09.803663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:09.803949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:09.804224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:09.812878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:09.814502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:09.823286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1070/2138] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 13:58:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:58:10.150696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:10.151264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:10.151546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:10.169722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:10.170915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:10.181577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1069/2139] 10.222.0.91 () {58 vars in 967 bytes} [Thu May 26 13:58:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:10.200145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:10.211396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118129f5-e925-471a-a91c-4410adfe7ec3 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:10.216678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e946fb-e41d-48ef-be82-accf1dfe16cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:10.217452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e946fb-e41d-48ef-be82-accf1dfe16cc service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:10.217838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e946fb-e41d-48ef-be82-accf1dfe16cc service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:10.218186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e946fb-e41d-48ef-be82-accf1dfe16cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:10.226652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e946fb-e41d-48ef-be82-accf1dfe16cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:10.228264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e946fb-e41d-48ef-be82-accf1dfe16cc service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:10.236948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1071/2140] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 13:58:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:58:17.596034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:17.597503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:17.662570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1c3b8-4fed-4149-96f8-647d271312ee None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:17.667071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:17.667935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:17.668214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:17.668488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] PATH_INFO: `/v3/users/f90654dfe0c94d2ab61872dd244f4d81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:17.670323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90654dfe0c94d2ab61872dd244f4d81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:17.672243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:17.720225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90654dfe0c94d2ab61872dd244f4d81'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:17.727912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90654dfe0c94d2ab61872dd244f4d81'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:17.741333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90654dfe0c94d2ab61872dd244f4d81'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:17.742015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90654dfe0c94d2ab61872dd244f4d81'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:17.742626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90654dfe0c94d2ab61872dd244f4d81'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:17.754832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1070/2141] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:17 2022] DELETE /identity/v3/users/f90654dfe0c94d2ab61872dd244f4d81 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:17.759919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:17.760402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:17.760495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:17.778755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:17.780508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:17.842334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1072/2142] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:17.851676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:17.853177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:17.893627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:17.913787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca58a0-88e9-4912-b859-8e801896c2cb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:17.918195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:17.918845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:17.919130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:17.919462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:17.930205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:17.931787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:17.943077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1071/2143] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:18.150400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:18.152158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:18.167516 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746fb88-f48f-43f0-8fe0-e02ca744b23c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:18.173339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:18.182514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:18.182803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:18.183088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] PATH_INFO: `/v3/projects/0926e6ea11a2454e9a5b4ef6c5fd2d96` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:18.185756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0926e6ea11a2454e9a5b4ef6c5fd2d96)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:18.187711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:18.243235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0926e6ea11a2454e9a5b4ef6c5fd2d96', 'request_id': 'req-50346f8c-a239-4380-a3a2-85f047e16e53'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:18.250639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0926e6ea11a2454e9a5b4ef6c5fd2d96', 'request_id': 'req-50346f8c-a239-4380-a3a2-85f047e16e53'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:18.257276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0926e6ea11a2454e9a5b4ef6c5fd2d96', 'request_id': 'req-50346f8c-a239-4380-a3a2-85f047e16e53'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:18.270525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] The token cache is being invalidate because project 0926e6ea11a2454e9a5b4ef6c5fd2d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:18.270833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:18.271468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:18.273048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1073/2144] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:18 2022] DELETE /identity/v3/projects/0926e6ea11a2454e9a5b4ef6c5fd2d96 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:21.057664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:21.059077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:21.118756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e530a-44e3-4de9-9314-e0b8adf38b7f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:21.167836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a008ebb-b5a2-44ed-9dff-5d7fbdaf409d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:21.168541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a008ebb-b5a2-44ed-9dff-5d7fbdaf409d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:21.168758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a008ebb-b5a2-44ed-9dff-5d7fbdaf409d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:21.169029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a008ebb-b5a2-44ed-9dff-5d7fbdaf409d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:21.170863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a008ebb-b5a2-44ed-9dff-5d7fbdaf409d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:21.234085 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a008ebb-b5a2-44ed-9dff-5d7fbdaf409d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:21.235592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a008ebb-b5a2-44ed-9dff-5d7fbdaf409d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:21.245211 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1072/2145] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:24.473788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:24.476095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:24.492484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:24.567001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50346f8c-a239-4380-a3a2-85f047e16e53 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:24.582184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:24.583065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:24.583343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:24.583616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] PATH_INFO: `/v3/users/9276b4d8fdf6484c80875bf4d151cb00` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:24.584823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9276b4d8fdf6484c80875bf4d151cb00)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:24.586800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:24.659261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9276b4d8fdf6484c80875bf4d151cb00'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:24.675256 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9276b4d8fdf6484c80875bf4d151cb00'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:24.676053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9276b4d8fdf6484c80875bf4d151cb00'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:24.676805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9276b4d8fdf6484c80875bf4d151cb00'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:24.686171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9276b4d8fdf6484c80875bf4d151cb00'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:24.699803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1074/2146] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:24 2022] DELETE /identity/v3/users/9276b4d8fdf6484c80875bf4d151cb00 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:24.705022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:24.705542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:24.705660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:24.724229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:24.725476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:24.770278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:24.787713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1073/2147] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:24.798619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:24.800117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:24.863479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c0b41-8cf2-4fcb-8f8e-aad1d481b65f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:24.867540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:24.868336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:24.868632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:24.868955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:24.879913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:24.881649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:24.893352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1075/2148] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:25.203176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:25.204590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:25.218846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53ebd5-8862-412e-bca0-fae6f25d659a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:25.228164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:25.229041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:25.229377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:25.230059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] PATH_INFO: `/v3/projects/fa35fd7465ae472184c6c05bb069c2fd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:25.230894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa35fd7465ae472184c6c05bb069c2fd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:25.233360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:25.280770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa35fd7465ae472184c6c05bb069c2fd', 'request_id': 'req-d3f1408d-a323-4839-8e58-165db4a0f096'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.288158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa35fd7465ae472184c6c05bb069c2fd', 'request_id': 'req-d3f1408d-a323-4839-8e58-165db4a0f096'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.294844 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa35fd7465ae472184c6c05bb069c2fd', 'request_id': 'req-d3f1408d-a323-4839-8e58-165db4a0f096'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.305940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] The token cache is being invalidate because project fa35fd7465ae472184c6c05bb069c2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:25.306421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.307053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.308681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1074/2149] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:25 2022] DELETE /identity/v3/projects/fa35fd7465ae472184c6c05bb069c2fd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:25.313496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:25.323283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61722f-1dd5-41cc-8a63-2b637c7cb786 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:25.354884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:25.383405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:25.384282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:25.384564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:25.384839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] PATH_INFO: `/v3/users/82e38f926d5a437ea35bbeb2a680c702` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:25.386059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e38f926d5a437ea35bbeb2a680c702)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:25.387987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:25.434231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e38f926d5a437ea35bbeb2a680c702'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.448262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e38f926d5a437ea35bbeb2a680c702'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.448965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e38f926d5a437ea35bbeb2a680c702'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.449598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e38f926d5a437ea35bbeb2a680c702'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.457096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e38f926d5a437ea35bbeb2a680c702'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.469869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1076/2150] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:25 2022] DELETE /identity/v3/users/82e38f926d5a437ea35bbeb2a680c702 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:25.749933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:25.751405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:25.765329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1408d-a323-4839-8e58-165db4a0f096 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:25.828795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:25.829709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:25.830010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:25.830284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] PATH_INFO: `/v3/projects/c9e89baac8414599a12858ba60ac9fd7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:25.831606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e89baac8414599a12858ba60ac9fd7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:25.833615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:25.881374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e89baac8414599a12858ba60ac9fd7', 'request_id': 'req-964bec5c-4b26-419e-bcd1-1899593a26a6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.890968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e89baac8414599a12858ba60ac9fd7', 'request_id': 'req-964bec5c-4b26-419e-bcd1-1899593a26a6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.897942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e89baac8414599a12858ba60ac9fd7', 'request_id': 'req-964bec5c-4b26-419e-bcd1-1899593a26a6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.909515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] The token cache is being invalidate because project c9e89baac8414599a12858ba60ac9fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:25.909811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.910471 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:25.912562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1075/2151] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:25 2022] DELETE /identity/v3/projects/c9e89baac8414599a12858ba60ac9fd7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:25.916693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:25.918198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:25.931454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cff3b-1508-4f40-b556-92d68f17818c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:25.989732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:25.990612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:25.990894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:25.991170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] PATH_INFO: `/v3/users/3684137ae5d04b7e9e68d7a4c74b2036` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:25.992359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3684137ae5d04b7e9e68d7a4c74b2036)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:25.994329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.055420 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3684137ae5d04b7e9e68d7a4c74b2036'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.062691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:26.068795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3684137ae5d04b7e9e68d7a4c74b2036'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.069890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3684137ae5d04b7e9e68d7a4c74b2036'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.070548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3684137ae5d04b7e9e68d7a4c74b2036'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.078690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3abf12af-e7fb-416d-ba95-88d3c6eab9d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3684137ae5d04b7e9e68d7a4c74b2036'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.092949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1077/2152] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:25 2022] DELETE /identity/v3/users/3684137ae5d04b7e9e68d7a4c74b2036 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:26.369388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:26.370825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:26.381434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:26.390060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bec5c-4b26-419e-bcd1-1899593a26a6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:26.452574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:26.453512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.453799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.454079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] PATH_INFO: `/v3/projects/ad8019ed81224ed088e2ed4ea724a994` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.455301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8019ed81224ed088e2ed4ea724a994)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:26.457285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.513873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8019ed81224ed088e2ed4ea724a994', 'request_id': 'req-39ddd177-dea6-4110-86a8-ea4c31c1ec01'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.523552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8019ed81224ed088e2ed4ea724a994', 'request_id': 'req-39ddd177-dea6-4110-86a8-ea4c31c1ec01'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.530554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8019ed81224ed088e2ed4ea724a994', 'request_id': 'req-39ddd177-dea6-4110-86a8-ea4c31c1ec01'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.540731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] The token cache is being invalidate because project ad8019ed81224ed088e2ed4ea724a994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:26.541019 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.541689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39ddd177-dea6-4110-86a8-ea4c31c1ec01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:26.543219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1076/2153] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:26 2022] DELETE /identity/v3/projects/ad8019ed81224ed088e2ed4ea724a994 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:26.549012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba317-a2db-4ed7-82aa-cece00820245 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.549335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba317-a2db-4ed7-82aa-cece00820245 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.549620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba317-a2db-4ed7-82aa-cece00820245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.577977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d4dba317-a2db-4ed7-82aa-cece00820245 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:26.579175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d4dba317-a2db-4ed7-82aa-cece00820245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:26.641702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1078/2154] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:26.650170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b158f5-9879-4d06-87a5-73d5587dd15f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.650701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b158f5-9879-4d06-87a5-73d5587dd15f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.651007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b158f5-9879-4d06-87a5-73d5587dd15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.669006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c1b158f5-9879-4d06-87a5-73d5587dd15f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:26.670249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c1b158f5-9879-4d06-87a5-73d5587dd15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:26.678242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1077/2155] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:26.682563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba317-a2db-4ed7-82aa-cece00820245 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:26.693265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba317-a2db-4ed7-82aa-cece00820245 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:26.697887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:26.700344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.700710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.701518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.702290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:58:26.702605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:26.704359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.717963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1079/2156] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:58:26.722521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b158f5-9879-4d06-87a5-73d5587dd15f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:26.732540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b158f5-9879-4d06-87a5-73d5587dd15f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:26.737499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:26.738203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.738443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.738722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.739439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:26.742762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.775733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1078/2157] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:26 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:26.780231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:26.790178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba6e5a-e063-476e-9dc5-27d46670d2dc admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:26.846073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:26.846930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.847227 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.847538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.848225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:26.850490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.871361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1080/2158] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:26 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:26.876090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:26.886283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58d7df-6e7b-42e0-a5aa-cc8811cd83fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:26.890831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:26.891480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.891761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.892034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.892905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:26.894534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.905167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1079/2159] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:26.908877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:26.918997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b68ca-1e9e-4eb7-8025-6b4d4abfd2a7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:26.924212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b62cb7-a4ca-4954-a280-47c7f19604a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:26.924879 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b62cb7-a4ca-4954-a280-47c7f19604a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.925189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b62cb7-a4ca-4954-a280-47c7f19604a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.925475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b62cb7-a4ca-4954-a280-47c7f19604a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.926224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b62cb7-a4ca-4954-a280-47c7f19604a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:26.927826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b62cb7-a4ca-4954-a280-47c7f19604a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.940490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1081/2160] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:26.944812 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:26.955046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bd822-fc9f-4c58-b704-1a0c8d779630 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:26.959683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:26.960557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:26.960850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:26.961121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] PATH_INFO: `/v3/projects/3ec8407e8ba2492187f4b2560810c4b0/users/a37a876cdf4a40eba4a38f1d02b5a44b/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:26.975570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec8407e8ba2492187f4b2560810c4b0, user_id=a37a876cdf4a40eba4a38f1d02b5a44b, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:26.978284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:26.991215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1080/2161] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:26 2022] PUT /identity/v3/projects/3ec8407e8ba2492187f4b2560810c4b0/users/a37a876cdf4a40eba4a38f1d02b5a44b/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:27.003157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022647-c189-4cf2-8a84-f8bb713ebd0c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.003388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022647-c189-4cf2-8a84-f8bb713ebd0c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.003667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022647-c189-4cf2-8a84-f8bb713ebd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.033957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-47022647-c189-4cf2-8a84-f8bb713ebd0c None None] MFA Rules not processed for user `a37a876cdf4a40eba4a38f1d02b5a44b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:27.035929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-47022647-c189-4cf2-8a84-f8bb713ebd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:27.059326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:27.117673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1082/2162] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:27.122170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.154832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:27.191551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2741f-e9cf-4615-bbfd-8bef8fe0ba03 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.197070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.197538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.197852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.198129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.198849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:27.200951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:27.236186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1081/2163] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:27 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:27.240374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47022647-c189-4cf2-8a84-f8bb713ebd0c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.252538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47022647-c189-4cf2-8a84-f8bb713ebd0c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.337978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.338680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.338967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.339247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.340016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:27.343772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:27.370600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1083/2164] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:27 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:27.374561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.384794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5be81a-7ec3-4ae9-a317-2ee4e17e5470 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.389348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.390010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.390288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.390566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.391289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:27.392879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:27.404606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1082/2165] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:27.408860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.419023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860e02b-caed-4cb3-91fa-ae4571333304 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.424309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.425216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.425503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.425778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] PATH_INFO: `/v3/projects/5fc822b2d5f04b188c46f77679f9e0b9/users/e175a3ec5e8241e08b8b9bebf75f5eb3/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.443235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc822b2d5f04b188c46f77679f9e0b9, user_id=e175a3ec5e8241e08b8b9bebf75f5eb3, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:27.445970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:27.461265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1084/2166] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:27 2022] PUT /identity/v3/projects/5fc822b2d5f04b188c46f77679f9e0b9/users/e175a3ec5e8241e08b8b9bebf75f5eb3/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:27.465294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.477198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165540d-617b-41ba-8884-395a1447a70b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.546095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228994c2-93d8-4e23-a32b-410ea996a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.546769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228994c2-93d8-4e23-a32b-410ea996a3db admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.547052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228994c2-93d8-4e23-a32b-410ea996a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.547323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228994c2-93d8-4e23-a32b-410ea996a3db admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.548043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228994c2-93d8-4e23-a32b-410ea996a3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:27.549700 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228994c2-93d8-4e23-a32b-410ea996a3db admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:27.560729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1083/2167] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:27.564676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.575142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb094dc5-a7ed-4add-8576-9ddab08416c3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.580715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.581690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.581976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.582254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] PATH_INFO: `/v3/projects/5fc822b2d5f04b188c46f77679f9e0b9/users/e175a3ec5e8241e08b8b9bebf75f5eb3/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.584398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc822b2d5f04b188c46f77679f9e0b9, user_id=e175a3ec5e8241e08b8b9bebf75f5eb3, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:27.587364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:27.602264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1085/2168] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:27 2022] PUT /identity/v3/projects/5fc822b2d5f04b188c46f77679f9e0b9/users/e175a3ec5e8241e08b8b9bebf75f5eb3/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:27.613595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80962825-23b4-4266-93a6-a6e7f705f6d3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.613889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80962825-23b4-4266-93a6-a6e7f705f6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.614165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80962825-23b4-4266-93a6-a6e7f705f6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.642450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-80962825-23b4-4266-93a6-a6e7f705f6d3 None None] MFA Rules not processed for user `e175a3ec5e8241e08b8b9bebf75f5eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:27.643668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-80962825-23b4-4266-93a6-a6e7f705f6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:27.714599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1084/2169] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:27.730139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.732013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:27.797395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75500-06ee-441e-a5ad-e4cee8f6617e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.801242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.801900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.802187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.802465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.818353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:27.820024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:27.830751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1086/2170] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:27.934371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80962825-23b4-4266-93a6-a6e7f705f6d3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:27.944249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80962825-23b4-4266-93a6-a6e7f705f6d3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:27.948024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b362ea-1a91-4978-a7a7-fc2746bbc814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:27.948609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b362ea-1a91-4978-a7a7-fc2746bbc814 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:27.948897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b362ea-1a91-4978-a7a7-fc2746bbc814 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:27.949176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b362ea-1a91-4978-a7a7-fc2746bbc814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:27.970515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:28.007190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b362ea-1a91-4978-a7a7-fc2746bbc814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:28.008636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b362ea-1a91-4978-a7a7-fc2746bbc814 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:28.018197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1085/2171] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:31.636309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:31.654566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:31.701078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807446a-05fa-4c6e-af96-f679c7f4260f service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:31.705704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:31.706575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:31.706859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:31.707136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] PATH_INFO: `/v3/users/a37a876cdf4a40eba4a38f1d02b5a44b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:31.708329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37a876cdf4a40eba4a38f1d02b5a44b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:31.710309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:31.770425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37a876cdf4a40eba4a38f1d02b5a44b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:31.782658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37a876cdf4a40eba4a38f1d02b5a44b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:31.783338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37a876cdf4a40eba4a38f1d02b5a44b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:31.783940 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37a876cdf4a40eba4a38f1d02b5a44b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:31.792421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37a876cdf4a40eba4a38f1d02b5a44b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:31.804964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1087/2172] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:31 2022] DELETE /identity/v3/users/a37a876cdf4a40eba4a38f1d02b5a44b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:31.809520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:31.810697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:31.810697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:31.829615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:31.830871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:31.890715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1086/2173] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:31.900781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:31.902271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:31.959873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494c931-52af-4c3e-a181-bd74bd499d80 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:31.963490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:31.964224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:31.964512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:31.964790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:31.975824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:31.977314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:31.993673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1088/2174] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:32.288469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:32.289988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:32.303745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7487a40-7c2b-422b-a8ed-94e50493a8a0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:32.309905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:32.311018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:32.311417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:32.311788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] PATH_INFO: `/v3/projects/3ec8407e8ba2492187f4b2560810c4b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:32.313054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec8407e8ba2492187f4b2560810c4b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:32.315009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:32.363113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec8407e8ba2492187f4b2560810c4b0', 'request_id': 'req-c906cd46-35d2-40fa-9d88-396ebe4fc040'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.370785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec8407e8ba2492187f4b2560810c4b0', 'request_id': 'req-c906cd46-35d2-40fa-9d88-396ebe4fc040'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.377989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec8407e8ba2492187f4b2560810c4b0', 'request_id': 'req-c906cd46-35d2-40fa-9d88-396ebe4fc040'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.390245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] The token cache is being invalidate because project 3ec8407e8ba2492187f4b2560810c4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:32.390531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.391185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.392775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1087/2175] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:32 2022] DELETE /identity/v3/projects/3ec8407e8ba2492187f4b2560810c4b0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:32.396876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:32.406723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa87d9-9f63-4d81-aea4-5da96bf60a89 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:32.441194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:32.466813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:32.467688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:32.467971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:32.468255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] PATH_INFO: `/v3/users/e175a3ec5e8241e08b8b9bebf75f5eb3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:32.469469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e175a3ec5e8241e08b8b9bebf75f5eb3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:32.471385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:32.516606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e175a3ec5e8241e08b8b9bebf75f5eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.529090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e175a3ec5e8241e08b8b9bebf75f5eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.529784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e175a3ec5e8241e08b8b9bebf75f5eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.530394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e175a3ec5e8241e08b8b9bebf75f5eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.537278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cde01c23-fdc3-4de8-af65-48717c34958c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e175a3ec5e8241e08b8b9bebf75f5eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.549756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1089/2176] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:32 2022] DELETE /identity/v3/users/e175a3ec5e8241e08b8b9bebf75f5eb3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:32.832816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:32.834958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:32.849683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906cd46-35d2-40fa-9d88-396ebe4fc040 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:32.902578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:32.916670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:32.917651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:32.917937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:32.918212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] PATH_INFO: `/v3/projects/5fc822b2d5f04b188c46f77679f9e0b9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:32.919923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc822b2d5f04b188c46f77679f9e0b9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:32.921878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:32.967998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc822b2d5f04b188c46f77679f9e0b9', 'request_id': 'req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.975731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc822b2d5f04b188c46f77679f9e0b9', 'request_id': 'req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.984701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc822b2d5f04b188c46f77679f9e0b9', 'request_id': 'req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.995987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] The token cache is being invalidate because project 5fc822b2d5f04b188c46f77679f9e0b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:32.996276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.996933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f6036aad-11c4-4e70-a6b4-6bd404d9eb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:32.999079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1088/2177] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:32 2022] DELETE /identity/v3/projects/5fc822b2d5f04b188c46f77679f9e0b9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:33.004155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2919781-1cfd-4580-bfcc-f23b4bb3861a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.004609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2919781-1cfd-4580-bfcc-f23b4bb3861a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.004697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2919781-1cfd-4580-bfcc-f23b4bb3861a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.023394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a2919781-1cfd-4580-bfcc-f23b4bb3861a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:33.024770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a2919781-1cfd-4580-bfcc-f23b4bb3861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:33.098491 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1090/2178] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:33.107270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13cff-60d0-4202-a602-815df27e9aad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.107577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13cff-60d0-4202-a602-815df27e9aad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.107852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13cff-60d0-4202-a602-815df27e9aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.130914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d7f13cff-60d0-4202-a602-815df27e9aad None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:33.132688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7f13cff-60d0-4202-a602-815df27e9aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:33.144395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1089/2179] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:33.148767 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2919781-1cfd-4580-bfcc-f23b4bb3861a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.160413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2919781-1cfd-4580-bfcc-f23b4bb3861a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.165396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.166056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.166374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.166652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.167382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:58:33.167694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.169921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.181804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1091/2180] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:58:33.186102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13cff-60d0-4202-a602-815df27e9aad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.198439 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13cff-60d0-4202-a602-815df27e9aad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.202558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.203227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.203507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.203778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.204515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.206450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.234980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1090/2181] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:33 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:33.238975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.249445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76e3d2-75bb-49a7-b46c-085bb9d53e6c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.299598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:33.317965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.318696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.318991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.319294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.320020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.322445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.346902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1092/2182] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:33 2022] POST /identity/v3/users => generated 305 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:33.348088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.359241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b983e-6ede-4281-914b-816ba2e95171 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.365979 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.366913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.367084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.367354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.367973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.370551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.382573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1091/2183] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:33.386257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.396308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5d434-d4c9-454f-ad61-38382211ccac admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.401490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed982ea-0162-4dd5-ba4c-e809981f8479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.402158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed982ea-0162-4dd5-ba4c-e809981f8479 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.402446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed982ea-0162-4dd5-ba4c-e809981f8479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.402727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed982ea-0162-4dd5-ba4c-e809981f8479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.403971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed982ea-0162-4dd5-ba4c-e809981f8479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.405623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed982ea-0162-4dd5-ba4c-e809981f8479 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.415500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1093/2184] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:33.420074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.432811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8853628-e24c-4dba-8de5-cb2dbabde56c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.437497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.438383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.438667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.438969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] PATH_INFO: `/v3/projects/d9cbd4010b864c59a2ded4bec118c2d5/users/5bf01df0c5b741f7a620dac98a6ec94f/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.458106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9cbd4010b864c59a2ded4bec118c2d5, user_id=5bf01df0c5b741f7a620dac98a6ec94f, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.462918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.478130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1092/2185] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:33 2022] PUT /identity/v3/projects/d9cbd4010b864c59a2ded4bec118c2d5/users/5bf01df0c5b741f7a620dac98a6ec94f/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:33.488155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c4d08-3959-4fd4-8351-1ac2d1d7fc17 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.488446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c4d08-3959-4fd4-8351-1ac2d1d7fc17 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.488721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c4d08-3959-4fd4-8351-1ac2d1d7fc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.515786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-058c4d08-3959-4fd4-8351-1ac2d1d7fc17 None None] MFA Rules not processed for user `5bf01df0c5b741f7a620dac98a6ec94f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:33.517048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-058c4d08-3959-4fd4-8351-1ac2d1d7fc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:33.582643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1094/2186] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:33 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:33.587714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.655052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39824dd9-3c77-4b72-9838-744da713cbf2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.659896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.660561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.660847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.661123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.661848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.663746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.685820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:33.696626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1093/2187] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:33 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:33.697826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c4d08-3959-4fd4-8351-1ac2d1d7fc17 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.709690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c4d08-3959-4fd4-8351-1ac2d1d7fc17 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.785909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.786619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.786932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.787263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.788655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.790146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.810733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1095/2188] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:33 2022] POST /identity/v3/users => generated 310 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:33.814838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.826116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd5369-83c8-4e26-856d-52fffbf0c13a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.830670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.831325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.831606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.831879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.832625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.834290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.845419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1094/2189] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:33.849512 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.860912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4d359-8338-406c-b011-a09775a26f7e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.866571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.867339 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.867539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.867812 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] PATH_INFO: `/v3/projects/677994e19cfe49579ceb63dfb7b6b36e/users/7ec03dd183e84b6dbf4944bbdc5b4d3d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.884647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=677994e19cfe49579ceb63dfb7b6b36e, user_id=7ec03dd183e84b6dbf4944bbdc5b4d3d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.887399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.901776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1096/2190] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:33 2022] PUT /identity/v3/projects/677994e19cfe49579ceb63dfb7b6b36e/users/7ec03dd183e84b6dbf4944bbdc5b4d3d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:33.906536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:33.916399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70ed60-27bd-46e8-9ae7-eef9917bf4ce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:33.973770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c7aaa-6678-4b39-a57b-501975a2ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:33.974434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c7aaa-6678-4b39-a57b-501975a2ada6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:33.974718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c7aaa-6678-4b39-a57b-501975a2ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:33.974994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c7aaa-6678-4b39-a57b-501975a2ada6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:33.975713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c7aaa-6678-4b39-a57b-501975a2ada6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:33.977325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c7aaa-6678-4b39-a57b-501975a2ada6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:33.987653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1095/2191] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:33.991868 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:34.002946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba4d52-cf9b-4cd5-9eb9-4012c9a6c9b6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:34.007686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:34.008599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:34.008883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:34.009181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] PATH_INFO: `/v3/projects/677994e19cfe49579ceb63dfb7b6b36e/users/7ec03dd183e84b6dbf4944bbdc5b4d3d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:34.011167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] RBAC: Authorizing `identity:create_grant(project_id=677994e19cfe49579ceb63dfb7b6b36e, user_id=7ec03dd183e84b6dbf4944bbdc5b4d3d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:34.014622 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:34.025063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1097/2192] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:33 2022] PUT /identity/v3/projects/677994e19cfe49579ceb63dfb7b6b36e/users/7ec03dd183e84b6dbf4944bbdc5b4d3d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:34.037804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7f41e-6ff8-4503-8cd0-8f847f264e89 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:34.038099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7f41e-6ff8-4503-8cd0-8f847f264e89 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:34.038384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7f41e-6ff8-4503-8cd0-8f847f264e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:34.066037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-11b7f41e-6ff8-4503-8cd0-8f847f264e89 None None] MFA Rules not processed for user `7ec03dd183e84b6dbf4944bbdc5b4d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:34.067342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-11b7f41e-6ff8-4503-8cd0-8f847f264e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:34.136525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1096/2193] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:34 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:34.149393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:34.151859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:34.156655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:34.236297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfab632-e064-48d6-b822-3d75c43a190c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:34.240025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:34.240677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:34.240963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:34.241311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:34.251054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:34.252495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:34.261916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1098/2194] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:34 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:34.421486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7f41e-6ff8-4503-8cd0-8f847f264e89 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:34.438482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7f41e-6ff8-4503-8cd0-8f847f264e89 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:34.443341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:34.443341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:34.443575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:34.443798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:34.508340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:34.510037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:34.520327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1097/2195] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:44.257742 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=33 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:44.515960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:55.230544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:55.253782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:55.255265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:55.297013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f61c7-add7-4e90-a729-2702c106efcf service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:55.301790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:55.302671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:55.302953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:55.303862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] PATH_INFO: `/v3/users/5bf01df0c5b741f7a620dac98a6ec94f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:55.305104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf01df0c5b741f7a620dac98a6ec94f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:55.307070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:55.335708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874e6be-94ea-4b1a-ae75-fc8bf275714b service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:55.343943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:55.345119 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:55.345668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:55.345957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] PATH_INFO: `/v3/users/784625a5936f44359533d9476e7f9061` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:55.349013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784625a5936f44359533d9476e7f9061)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:55.351858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:55.369547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf01df0c5b741f7a620dac98a6ec94f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.384362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf01df0c5b741f7a620dac98a6ec94f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.385100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf01df0c5b741f7a620dac98a6ec94f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.385987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf01df0c5b741f7a620dac98a6ec94f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.399962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2443e9d2-9e2e-476e-9613-f3ad1b145632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf01df0c5b741f7a620dac98a6ec94f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.418027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1099/2196] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:55 2022] DELETE /identity/v3/users/5bf01df0c5b741f7a620dac98a6ec94f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:55.422623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:55.422991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:55.423342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:55.428482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784625a5936f44359533d9476e7f9061'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.440444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784625a5936f44359533d9476e7f9061'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.444820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:55.446876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:55.456299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784625a5936f44359533d9476e7f9061'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.457002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784625a5936f44359533d9476e7f9061'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.457714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-61b2b6f5-12ab-4e57-9351-097bedfe5943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784625a5936f44359533d9476e7f9061'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:55.470783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1098/2197] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:55 2022] DELETE /identity/v3/users/784625a5936f44359533d9476e7f9061 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:58:55.475334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:55.475543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:55.475897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:55.497709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:55.498927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:55.510687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1100/2198] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:55.520184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:55.521543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:55.548558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:55.572229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1099/2199] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:55.581601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:55.583214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:55.594832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371754df-10e5-426f-8d58-0680eb12f2bd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:55.612374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:55.657247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:55.657951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:55.658234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:55.658509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:55.658609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c0f0e-306f-46ee-88fe-ed655f84684a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:55.662351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:55.663145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:55.663528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:55.663906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:55.668560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:55.670084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:55.675161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:55.676839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:55.680833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1101/2200] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:58:55.688509 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1100/2201] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:56.101369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.103232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:56.127700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6044f-255a-48ce-b808-0430e7503a57 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.134685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.135569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.135854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.136163 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] PATH_INFO: `/v3/projects/d9cbd4010b864c59a2ded4bec118c2d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.137958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9cbd4010b864c59a2ded4bec118c2d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.139963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.222571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9cbd4010b864c59a2ded4bec118c2d5', 'request_id': 'req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.232715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9cbd4010b864c59a2ded4bec118c2d5', 'request_id': 'req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.238965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9cbd4010b864c59a2ded4bec118c2d5', 'request_id': 'req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.255501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] The token cache is being invalidate because project d9cbd4010b864c59a2ded4bec118c2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:56.255930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.256642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.258282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1102/2202] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:56 2022] DELETE /identity/v3/projects/d9cbd4010b864c59a2ded4bec118c2d5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:56.262677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.273314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285038b5-7ebd-4ba3-877e-41ce34cbc883 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.282466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.284194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:56.357532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.358423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.358710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.359318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] PATH_INFO: `/v3/users/7ec03dd183e84b6dbf4944bbdc5b4d3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.360811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec03dd183e84b6dbf4944bbdc5b4d3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.363857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.384936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1c98-09d7-4c0f-b550-e74ac1ffada7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.390455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.392617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.392907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.393209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] PATH_INFO: `/v3/projects/a725b19ff51c4cc9ac71f8a8cced6b1b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.394619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a725b19ff51c4cc9ac71f8a8cced6b1b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.396826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.399929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:56.426400 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec03dd183e84b6dbf4944bbdc5b4d3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.435886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec03dd183e84b6dbf4944bbdc5b4d3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.452027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec03dd183e84b6dbf4944bbdc5b4d3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.452767 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec03dd183e84b6dbf4944bbdc5b4d3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.453446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20b8e790-d814-4709-b00e-d2610aaf9a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec03dd183e84b6dbf4944bbdc5b4d3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.460253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a725b19ff51c4cc9ac71f8a8cced6b1b', 'request_id': 'req-366240cd-6df3-40d4-9b60-05320a8cb391'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.468980 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a725b19ff51c4cc9ac71f8a8cced6b1b', 'request_id': 'req-366240cd-6df3-40d4-9b60-05320a8cb391'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.475353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1101/2203] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:56 2022] DELETE /identity/v3/users/7ec03dd183e84b6dbf4944bbdc5b4d3d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:56.476100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a725b19ff51c4cc9ac71f8a8cced6b1b', 'request_id': 'req-366240cd-6df3-40d4-9b60-05320a8cb391'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.494324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] The token cache is being invalidate because project a725b19ff51c4cc9ac71f8a8cced6b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:56.494595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.495403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-366240cd-6df3-40d4-9b60-05320a8cb391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.500790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1103/2204] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:56 2022] DELETE /identity/v3/projects/a725b19ff51c4cc9ac71f8a8cced6b1b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:56.502683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078dc15-f437-4119-943d-afc7086d48e6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.502932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078dc15-f437-4119-943d-afc7086d48e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.503343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078dc15-f437-4119-943d-afc7086d48e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.522999 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0078dc15-f437-4119-943d-afc7086d48e6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:56.524444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0078dc15-f437-4119-943d-afc7086d48e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:56.576860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:56.623274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1102/2205] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:56.631614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879720c1-62ab-4567-9aa8-6f8dcd2ff514 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.631908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879720c1-62ab-4567-9aa8-6f8dcd2ff514 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.632231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879720c1-62ab-4567-9aa8-6f8dcd2ff514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.662429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-879720c1-62ab-4567-9aa8-6f8dcd2ff514 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:56.663900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-879720c1-62ab-4567-9aa8-6f8dcd2ff514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:56.672931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1104/2206] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:56.677044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078dc15-f437-4119-943d-afc7086d48e6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.688870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078dc15-f437-4119-943d-afc7086d48e6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.694253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.694819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.695169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.695961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.696314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:58:56.697023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.698745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.712153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1103/2207] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:58:56.716538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879720c1-62ab-4567-9aa8-6f8dcd2ff514 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.725900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879720c1-62ab-4567-9aa8-6f8dcd2ff514 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.730757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.731961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.732248 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.732527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.733271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.735193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.765194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1105/2208] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:56 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:56.769341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.780016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536377f4-3ab7-421d-b0ea-5787b9738b31 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.811549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.813321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:56.848140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.848866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.849359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.849644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.850373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.852420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.876839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1104/2209] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:56 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:56.877977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.887671 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ae790-a46e-4851-8b80-1f7b383d4c58 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.887922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2012b45-c044-42ac-98f1-b160cd17e885 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.892553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.892694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.893253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.893531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.893640 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.893806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.893913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.894080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] PATH_INFO: `/v3/projects/677994e19cfe49579ceb63dfb7b6b36e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.894526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.895262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=677994e19cfe49579ceb63dfb7b6b36e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.896115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.897202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.909518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1105/2210] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:56.910507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.920663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc47c-d338-413d-8afa-45f8d546c052 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.925798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:56.926918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.927208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.927644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:56.928498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:56.930327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:56.942096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1106/2211] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:56.946063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '677994e19cfe49579ceb63dfb7b6b36e', 'request_id': 'req-24b4bae9-663e-421b-a7c5-6cb08dab0855'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.946228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:56.952670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '677994e19cfe49579ceb63dfb7b6b36e', 'request_id': 'req-24b4bae9-663e-421b-a7c5-6cb08dab0855'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.956262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84827de-7426-42dd-ae22-31bfde27d675 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:56.958620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '677994e19cfe49579ceb63dfb7b6b36e', 'request_id': 'req-24b4bae9-663e-421b-a7c5-6cb08dab0855'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.961184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:56.975184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] The token cache is being invalidate because project 677994e19cfe49579ceb63dfb7b6b36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:56.975700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.976361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-24b4bae9-663e-421b-a7c5-6cb08dab0855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:56.978053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1106/2212] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:56 2022] DELETE /identity/v3/projects/677994e19cfe49579ceb63dfb7b6b36e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:58:56.983088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c4c0f-0b18-4c48-b8df-3cbcc0a548eb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:56.983373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c4c0f-0b18-4c48-b8df-3cbcc0a548eb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:56.983648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c4c0f-0b18-4c48-b8df-3cbcc0a548eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.002068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-177c4c0f-0b18-4c48-b8df-3cbcc0a548eb None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:57.003264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-177c4c0f-0b18-4c48-b8df-3cbcc0a548eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.017127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2e689-978c-4cd8-a93e-8e976f1926a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.018046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2e689-978c-4cd8-a93e-8e976f1926a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.018328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2e689-978c-4cd8-a93e-8e976f1926a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.018604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2e689-978c-4cd8-a93e-8e976f1926a6 admin admin] PATH_INFO: `/v3/projects/12082647641d4ece974c66384e3bf169/users/2eb759cf823b45d2aa35fdcacc7ac3e8/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.034934 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2e689-978c-4cd8-a93e-8e976f1926a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12082647641d4ece974c66384e3bf169, user_id=2eb759cf823b45d2aa35fdcacc7ac3e8, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.038182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2e689-978c-4cd8-a93e-8e976f1926a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.053163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1107/2213] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:56 2022] PUT /identity/v3/projects/12082647641d4ece974c66384e3bf169/users/2eb759cf823b45d2aa35fdcacc7ac3e8/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:57.058143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.058547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.058928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.066442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1107/2214] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:57.074298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40756c-4455-4a53-8326-d00113b0aa69 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.074834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40756c-4455-4a53-8326-d00113b0aa69 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.075116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40756c-4455-4a53-8326-d00113b0aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.077067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:57.078969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.092690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:57.094511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5e40756c-4455-4a53-8326-d00113b0aa69 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:57.095714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5e40756c-4455-4a53-8326-d00113b0aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.147045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1108/2215] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:57.158470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.159837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.160223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1108/2216] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:57.164681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40756c-4455-4a53-8326-d00113b0aa69 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.175817 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40756c-4455-4a53-8326-d00113b0aa69 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.180594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.181329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.181631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.181911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.182637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:58:57.182939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.184553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.197031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1109/2217] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:58:57.201664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.211831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56ae0-01ac-4a62-99f5-6e5d147244f5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.217343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.218085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.218418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.218805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.219550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.221765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.228632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66a314-40ba-46ea-9de8-3724ef237612 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.232879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.233601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.233891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.234172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.244737 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.246237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.252757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1110/2218] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:57.257208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.260873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1109/2219] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:58:57.325421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1222d-2fec-4bc5-a4a9-bbfa3cb148e1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.330463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.331191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.331476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.331754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.332448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.334449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.342139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:57.364528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.369343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1111/2220] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:57.375260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30268be8-01ad-4ef8-b35f-35d5dd17af90 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.380192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.381185 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.381304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.382719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.382719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.384299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.398761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1110/2221] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:58:57.400440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.415547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634f241-cf11-4a69-8146-c36980bb23d7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.421668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50c52f-7e7b-4592-b2de-3419471a8a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.422449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50c52f-7e7b-4592-b2de-3419471a8a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.422806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50c52f-7e7b-4592-b2de-3419471a8a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.423327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50c52f-7e7b-4592-b2de-3419471a8a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.423965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50c52f-7e7b-4592-b2de-3419471a8a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.425676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50c52f-7e7b-4592-b2de-3419471a8a44 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.444884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.446103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1112/2222] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:58:57.458146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de80241-08e0-449e-84ce-d86a9ed8626d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.466374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.467535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.467829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.468653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] PATH_INFO: `/v3/projects/42d862aaeaaf483eb1674dd3c23d353d/users/9ad50dcbd59346acadb79183f9ad500b/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.492843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d862aaeaaf483eb1674dd3c23d353d, user_id=9ad50dcbd59346acadb79183f9ad500b, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.495945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.513519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1111/2223] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:57 2022] PUT /identity/v3/projects/42d862aaeaaf483eb1674dd3c23d353d/users/9ad50dcbd59346acadb79183f9ad500b/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:57.525192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.525408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.525803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.556472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] MFA Rules not processed for user `9ad50dcbd59346acadb79183f9ad500b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:57.557719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.634897 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1113/2224] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:57.639192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.684381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.688262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.712130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63789fec-4cf8-468e-b4c1-b9c9a92dd95a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.717913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.718655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.719026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.719369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.720147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.722319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.751250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1112/2225] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:57.755223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.763765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a298-80c9-4d90-824d-a86ead00dc4b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.773115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:57.841831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fced58d-6deb-413e-89d0-e4b0149c4b70 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.842849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.843957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.844245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.844528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] PATH_INFO: `/v3/users/b499deffb3164de396174e39db5e932d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.846469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b499deffb3164de396174e39db5e932d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.849049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.851361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.852656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.853329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.853968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.855537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.859375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.869755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:57.890789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1113/2226] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:58:57.896755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.909932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b499deffb3164de396174e39db5e932d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:57.920716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c7bfd-71a2-44b3-9c90-fe0ab1978e5b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:57.924103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b499deffb3164de396174e39db5e932d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:57.924829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b499deffb3164de396174e39db5e932d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:57.925623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b499deffb3164de396174e39db5e932d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:57.928713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:57.929895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.930407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.930911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.932324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:57.935719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6c52e0f2-5146-43ee-832f-1e771f50f0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b499deffb3164de396174e39db5e932d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:57.936092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:57.953988 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1114/2227] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:58:57 2022] DELETE /identity/v3/users/b499deffb3164de396174e39db5e932d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:58:57.955570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38e5f4-d48a-4873-8342-ea4bb10c0d95 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:57.955949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38e5f4-d48a-4873-8342-ea4bb10c0d95 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:57.956239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38e5f4-d48a-4873-8342-ea4bb10c0d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:57.961384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1114/2228] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:58:57.966190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:57.968548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.976360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0c38e5f4-d48a-4873-8342-ea4bb10c0d95 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:57.977609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0c38e5f4-d48a-4873-8342-ea4bb10c0d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:57.983042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ea667-f64e-445b-87fe-e69a48c1ee84 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.079511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.080617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.081086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.081555 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.083514 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.086029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.090814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1115/2229] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:58.094080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bb215-9032-418b-9cb3-8e66ef642c77 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.094080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bb215-9032-418b-9cb3-8e66ef642c77 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.094080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bb215-9032-418b-9cb3-8e66ef642c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.105328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1115/2230] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:58.105328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.107442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.125127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-de7bb215-9032-418b-9cb3-8e66ef642c77 None None] MFA Rules not processed for user `2eb759cf823b45d2aa35fdcacc7ac3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:58.126726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-de7bb215-9032-418b-9cb3-8e66ef642c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.182645 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15420ce3-72bc-4d31-afdb-c37d3393d92f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.188103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.188835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.189361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.194638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.210851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.212534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.219997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1116/2231] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:58.221215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bb215-9032-418b-9cb3-8e66ef642c77 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.234719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bb215-9032-418b-9cb3-8e66ef642c77 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.237139 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1116/2232] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:58.238004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.239468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.240157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887b961-d554-4233-9188-c945b43575fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.241266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887b961-d554-4233-9188-c945b43575fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.241720 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887b961-d554-4233-9188-c945b43575fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.242725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887b961-d554-4233-9188-c945b43575fd admin admin] PATH_INFO: `/v3/projects/9a78ec335db2485b8bdc30bc906a0af1/users/040227d9f48e4b778f2d6ddc0f6935eb/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.260293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887b961-d554-4233-9188-c945b43575fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a78ec335db2485b8bdc30bc906a0af1, user_id=040227d9f48e4b778f2d6ddc0f6935eb, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.264074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887b961-d554-4233-9188-c945b43575fd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.274350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:58.281378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1117/2233] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:58 2022] PUT /identity/v3/projects/9a78ec335db2485b8bdc30bc906a0af1/users/040227d9f48e4b778f2d6ddc0f6935eb/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:58.282084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:58.291511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca8517-7f18-42e7-8ad5-3d9092e849c6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.292455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca8517-7f18-42e7-8ad5-3d9092e849c6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.292741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca8517-7f18-42e7-8ad5-3d9092e849c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.329137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-30ca8517-7f18-42e7-8ad5-3d9092e849c6 None None] MFA Rules not processed for user `040227d9f48e4b778f2d6ddc0f6935eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:58.330807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-30ca8517-7f18-42e7-8ad5-3d9092e849c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.333466 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5476d8-aa4f-46b9-b195-d990ef4f66ba service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.408671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1118/2234] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:58.413225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca8517-7f18-42e7-8ad5-3d9092e849c6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.419534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.420743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.421224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.421666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.489748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca8517-7f18-42e7-8ad5-3d9092e849c6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.492924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.494597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.495557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.496256 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.496573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.496850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.497565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.499475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.507927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1117/2235] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:58.539780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1119/2236] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:58 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:58.544163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.545059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.545977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.566330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529ca4-5afd-4619-884e-198d5d93b703 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.641030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e230eb-6a39-4a03-98ae-d39867829d2c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.648978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.650347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.650668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.650951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.651952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.654977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.671459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.672542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.672826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.673097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] PATH_INFO: `/v3/projects/fea63085a6f64135af82f0f671608c54` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.676091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea63085a6f64135af82f0f671608c54)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.678207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.691528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1120/2237] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:58 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:58.694193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.703530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:58.709064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6265d-96a3-4343-814b-4c0f1f50862e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.720834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.725382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.725640 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.727017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.727763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.729757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.743459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1121/2238] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:58.744906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.750358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:58.758060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea63085a6f64135af82f0f671608c54', 'request_id': 'req-3f9f1288-6a2d-4610-b027-2b68eec17b1b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:58.759735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a5f3e-e266-4589-807e-b6d1b67fc840 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:58.771851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea63085a6f64135af82f0f671608c54', 'request_id': 'req-3f9f1288-6a2d-4610-b027-2b68eec17b1b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:58.783066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea63085a6f64135af82f0f671608c54', 'request_id': 'req-3f9f1288-6a2d-4610-b027-2b68eec17b1b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:58.794584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] The token cache is being invalidate because project fea63085a6f64135af82f0f671608c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:58:58.794869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:58.795562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f9f1288-6a2d-4610-b027-2b68eec17b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:58:58.797131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1118/2239] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:58:58 2022] DELETE /identity/v3/projects/fea63085a6f64135af82f0f671608c54 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:58.801937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee5f4-0c23-42a8-9aa9-61dc36d4f1e3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.802353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee5f4-0c23-42a8-9aa9-61dc36d4f1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.802637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee5f4-0c23-42a8-9aa9-61dc36d4f1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.822595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-718ee5f4-0c23-42a8-9aa9-61dc36d4f1e3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:58.823828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-718ee5f4-0c23-42a8-9aa9-61dc36d4f1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.862850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:58.864161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.865425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.865604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] PATH_INFO: `/v3/projects/71a9f4d1848245f78d10ab85e6461167/users/252f4a1452004c38915e4b57bdff578a/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.889077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a9f4d1848245f78d10ab85e6461167, user_id=252f4a1452004c38915e4b57bdff578a, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:58.891806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:58.898320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1119/2240] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:58.907576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0053894-676b-4a48-b18d-02260af3db50 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.907872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0053894-676b-4a48-b18d-02260af3db50 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.908274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0053894-676b-4a48-b18d-02260af3db50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.915837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1122/2241] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:58 2022] PUT /identity/v3/projects/71a9f4d1848245f78d10ab85e6461167/users/252f4a1452004c38915e4b57bdff578a/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:58.919745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:58.921885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.930925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c0053894-676b-4a48-b18d-02260af3db50 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:58.932069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c0053894-676b-4a48-b18d-02260af3db50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.996365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8caf-b16d-451a-8261-9a20c93b5b44 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.000342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1120/2242] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:59.001330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0053894-676b-4a48-b18d-02260af3db50 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.002350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.003045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.003334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.003613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.004343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.006007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.010372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0053894-676b-4a48-b18d-02260af3db50 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.015867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.016372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1123/2243] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:58:59.016582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.016934 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.017244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.017981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:58:59.018288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.019970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.020175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.030080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2212c-b868-45c3-8ca3-ef58e42ceaf1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.036371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1121/2244] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:58:59.036544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd6730-777f-406c-b171-7fd2c739cc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.036544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd6730-777f-406c-b171-7fd2c739cc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.036544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd6730-777f-406c-b171-7fd2c739cc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.036544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd6730-777f-406c-b171-7fd2c739cc9d admin admin] PATH_INFO: `/v3/projects/71a9f4d1848245f78d10ab85e6461167/users/252f4a1452004c38915e4b57bdff578a/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.037908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.039067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd6730-777f-406c-b171-7fd2c739cc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a9f4d1848245f78d10ab85e6461167, user_id=252f4a1452004c38915e4b57bdff578a, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.041779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd6730-777f-406c-b171-7fd2c739cc9d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.048658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991cf8-08b1-4ccd-bf84-77c247cb26df admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.053342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1124/2245] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:59 2022] PUT /identity/v3/projects/71a9f4d1848245f78d10ab85e6461167/users/252f4a1452004c38915e4b57bdff578a/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:58:59.064416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.064785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.065070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.092540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] MFA Rules not processed for user `252f4a1452004c38915e4b57bdff578a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:59.094026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:59.107916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.108582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.108868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.109144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.110042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.112111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.115350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:59.147496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1122/2246] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:58:59 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:59.149002 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.151294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:59.181781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1125/2247] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:59.194363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.196321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:59.218619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd9c5-b49c-41c8-ab18-aed350502398 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.223232 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.223893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.224174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.224431 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.225289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.227191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.246397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1123/2248] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:58:59 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:58:59.250332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.258939 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a32be-5de0-45be-bbdf-354249e73ca1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.259739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69e972-49b7-4c4e-830c-b7dd9d549da1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.264799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.265496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.265776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.266048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.266769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.268346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.279696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1124/2249] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:58:59.283415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.292881 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157671a4-24e8-4396-a138-edbfdd22ab4b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.297602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.298316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.298667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.299012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.299796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.301468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.310789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.311518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.311870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.312130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1125/2250] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:58:59.312248 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.316114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.326355 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9efb73-3adf-48c4-b291-635c279f3ea2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.331029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02c25d-9ae7-4dc7-a360-63306ca884fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.331880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c25d-9ae7-4dc7-a360-63306ca884fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.332162 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c25d-9ae7-4dc7-a360-63306ca884fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.332436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c25d-9ae7-4dc7-a360-63306ca884fa admin admin] PATH_INFO: `/v3/projects/5ad7e1c6adb0483397b3b2de2ac60ec8/users/36e6babf99e24bcdb233ec89a27771f2/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.348196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02c25d-9ae7-4dc7-a360-63306ca884fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad7e1c6adb0483397b3b2de2ac60ec8, user_id=36e6babf99e24bcdb233ec89a27771f2, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.350989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02c25d-9ae7-4dc7-a360-63306ca884fa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.366527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1126/2251] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:58:59 2022] PUT /identity/v3/projects/5ad7e1c6adb0483397b3b2de2ac60ec8/users/36e6babf99e24bcdb233ec89a27771f2/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:58:59.377262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e675-60f1-4250-9f36-d53d76ac4517 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.377556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e675-60f1-4250-9f36-d53d76ac4517 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.377833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e675-60f1-4250-9f36-d53d76ac4517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.384555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.386039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.397048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1126/2252] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:58:59.406089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b856e675-60f1-4250-9f36-d53d76ac4517 None None] MFA Rules not processed for user `36e6babf99e24bcdb233ec89a27771f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:59.407422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b856e675-60f1-4250-9f36-d53d76ac4517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:59.441405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.442833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:59.454514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c5cdf-9624-4816-9545-7f2eea975e9b service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.475885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1127/2253] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:58:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:58:59.486231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856e675-60f1-4250-9f36-d53d76ac4517 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:59.504482 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.505137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.505465 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.505762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.507042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:59.507583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:59.513478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:59.546870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856e675-60f1-4250-9f36-d53d76ac4517 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:59.551383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e3f92-b37e-4218-8214-2bd41607e7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:59.552401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e3f92-b37e-4218-8214-2bd41607e7dd service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:59.552766 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e3f92-b37e-4218-8214-2bd41607e7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:59.553226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e3f92-b37e-4218-8214-2bd41607e7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:59.562574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e3f92-b37e-4218-8214-2bd41607e7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.564014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e3f92-b37e-4218-8214-2bd41607e7dd service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.569510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:59.570937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:59.573213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1128/2254] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:58:59.581628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1127/2255] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:58:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:59:01.037273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397cf43-1f20-4eff-a4c5-bf0766fe247f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:01.037712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397cf43-1f20-4eff-a4c5-bf0766fe247f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:01.037712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397cf43-1f20-4eff-a4c5-bf0766fe247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:01.061934 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1397cf43-1f20-4eff-a4c5-bf0766fe247f None None] MFA Rules not processed for user `2eb759cf823b45d2aa35fdcacc7ac3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:01.063268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1397cf43-1f20-4eff-a4c5-bf0766fe247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:01.120885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1129/2256] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:09.577326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=25 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:09.941751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:10.009506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e999ad-8694-4123-84d3-0a31301180dc service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:10.015218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:10.016204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:10.016480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:10.016755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] PATH_INFO: `/v3/users/9ad50dcbd59346acadb79183f9ad500b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:10.018119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad50dcbd59346acadb79183f9ad500b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:10.020055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:10.070486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad50dcbd59346acadb79183f9ad500b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.083043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad50dcbd59346acadb79183f9ad500b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.083830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad50dcbd59346acadb79183f9ad500b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.084348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad50dcbd59346acadb79183f9ad500b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.092091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad50dcbd59346acadb79183f9ad500b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.106028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1128/2257] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:59:09 2022] DELETE /identity/v3/users/9ad50dcbd59346acadb79183f9ad500b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:10.110533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2c63-fcf9-493f-a0ed-c92835ebc198 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:10.111083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2c63-fcf9-493f-a0ed-c92835ebc198 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:10.111189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2c63-fcf9-493f-a0ed-c92835ebc198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:10.129026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-93ba2c63-fcf9-493f-a0ed-c92835ebc198 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:10.130273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93ba2c63-fcf9-493f-a0ed-c92835ebc198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:10.206199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1130/2258] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:10.215589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:10.217044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:10.278230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7399e8-09e8-4eba-9937-d06311784104 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:10.283792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:10.284578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:10.284927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:10.285318 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:10.297380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:10.298848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:10.299540 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e0f2d-efb5-4938-9a74-23e1254ef8a5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:10.299838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e0f2d-efb5-4938-9a74-23e1254ef8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:10.300116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e0f2d-efb5-4938-9a74-23e1254ef8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:10.309442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1129/2259] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:59:10.318977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a89e0f2d-efb5-4938-9a74-23e1254ef8a5 None None] MFA Rules not processed for user `8d7bcc48c8d844e5b36b1c4e2748e766`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:10.320191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a89e0f2d-efb5-4938-9a74-23e1254ef8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:10.380401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1131/2260] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:59:10.604033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:10.605659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:10.633063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88b247-34f1-410a-bb15-3d93f74124c9 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:10.638856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:10.639728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:10.640014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:10.640293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] PATH_INFO: `/v3/projects/42d862aaeaaf483eb1674dd3c23d353d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:10.641530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d862aaeaaf483eb1674dd3c23d353d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:10.643550 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:10.661236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:10.694568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d862aaeaaf483eb1674dd3c23d353d', 'request_id': 'req-739094e9-3902-44e0-9d88-780ed727905b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.701541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d862aaeaaf483eb1674dd3c23d353d', 'request_id': 'req-739094e9-3902-44e0-9d88-780ed727905b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.707352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d862aaeaaf483eb1674dd3c23d353d', 'request_id': 'req-739094e9-3902-44e0-9d88-780ed727905b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.722825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] The token cache is being invalidate because project 42d862aaeaaf483eb1674dd3c23d353d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:59:10.722825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.723501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.725824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1130/2261] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:59:10 2022] DELETE /identity/v3/projects/42d862aaeaaf483eb1674dd3c23d353d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:10.729492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e0f2d-efb5-4938-9a74-23e1254ef8a5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:10.740732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e0f2d-efb5-4938-9a74-23e1254ef8a5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:10.751081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:10.802707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:10.803621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:10.803905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:10.804213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] PATH_INFO: `/v3/users/040227d9f48e4b778f2d6ddc0f6935eb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:10.805453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=040227d9f48e4b778f2d6ddc0f6935eb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:10.807376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:10.859215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '040227d9f48e4b778f2d6ddc0f6935eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.868684 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '040227d9f48e4b778f2d6ddc0f6935eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.881542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '040227d9f48e4b778f2d6ddc0f6935eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.882140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '040227d9f48e4b778f2d6ddc0f6935eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.882754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '040227d9f48e4b778f2d6ddc0f6935eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:10.895383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1132/2262] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:59:10 2022] DELETE /identity/v3/users/040227d9f48e4b778f2d6ddc0f6935eb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:11.216524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:11.218365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:11.234589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739094e9-3902-44e0-9d88-780ed727905b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:11.300983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:11.302073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:11.302400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:11.302678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] PATH_INFO: `/v3/projects/9a78ec335db2485b8bdc30bc906a0af1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:11.304044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a78ec335db2485b8bdc30bc906a0af1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:11.306207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:11.364749 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a78ec335db2485b8bdc30bc906a0af1', 'request_id': 'req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.371600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a78ec335db2485b8bdc30bc906a0af1', 'request_id': 'req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.377564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a78ec335db2485b8bdc30bc906a0af1', 'request_id': 'req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.391927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] The token cache is being invalidate because project 9a78ec335db2485b8bdc30bc906a0af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:59:11.391927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.392158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.394337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1131/2263] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:59:11 2022] DELETE /identity/v3/projects/9a78ec335db2485b8bdc30bc906a0af1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:11.398336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:11.400343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:11.414622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0c9c0-b229-463b-9343-3e58fa5a5553 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:11.473320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:11.474190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:11.474467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:11.474734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] PATH_INFO: `/v3/users/252f4a1452004c38915e4b57bdff578a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:11.475929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=252f4a1452004c38915e4b57bdff578a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:11.477932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:11.514557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:11.529405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252f4a1452004c38915e4b57bdff578a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.536271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252f4a1452004c38915e4b57bdff578a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.548843 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252f4a1452004c38915e4b57bdff578a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.549505 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252f4a1452004c38915e4b57bdff578a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.550115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-280394fa-bfac-478e-bced-fcc7a5cfbc86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252f4a1452004c38915e4b57bdff578a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.562304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1133/2264] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:59:11 2022] DELETE /identity/v3/users/252f4a1452004c38915e4b57bdff578a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:11.838757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:11.840878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:11.856667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea4df-fe93-4fca-a35d-d94d0c65c65a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:11.891233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:11.918214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:11.919008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:11.919292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:11.919595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] PATH_INFO: `/v3/projects/71a9f4d1848245f78d10ab85e6461167` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:11.920797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a9f4d1848245f78d10ab85e6461167)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:11.922780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:11.971862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a9f4d1848245f78d10ab85e6461167', 'request_id': 'req-06f491fe-367b-4ab0-a661-b248b6937be5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.978054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a9f4d1848245f78d10ab85e6461167', 'request_id': 'req-06f491fe-367b-4ab0-a661-b248b6937be5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.983756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a9f4d1848245f78d10ab85e6461167', 'request_id': 'req-06f491fe-367b-4ab0-a661-b248b6937be5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.996337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] The token cache is being invalidate because project 71a9f4d1848245f78d10ab85e6461167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:59:11.996672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.997329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06f491fe-367b-4ab0-a661-b248b6937be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:11.998953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1132/2265] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:59:11 2022] DELETE /identity/v3/projects/71a9f4d1848245f78d10ab85e6461167 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:12.004797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5bc43-8d42-4278-b7be-05ee47cb1063 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.005087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5bc43-8d42-4278-b7be-05ee47cb1063 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.005442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5bc43-8d42-4278-b7be-05ee47cb1063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.023808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9ae5bc43-8d42-4278-b7be-05ee47cb1063 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:12.025282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ae5bc43-8d42-4278-b7be-05ee47cb1063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:12.084978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1134/2266] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:12.094443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213c5ce-809e-4a32-a84a-5cbccf6842d0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.094736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213c5ce-809e-4a32-a84a-5cbccf6842d0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.095019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213c5ce-809e-4a32-a84a-5cbccf6842d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.114433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6213c5ce-809e-4a32-a84a-5cbccf6842d0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:12.115648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6213c5ce-809e-4a32-a84a-5cbccf6842d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:12.124396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1133/2267] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:59:12.128678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5bc43-8d42-4278-b7be-05ee47cb1063 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:12.139146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5bc43-8d42-4278-b7be-05ee47cb1063 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:12.143781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:12.144488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.144780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.145057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.145822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:59:12.146266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:12.147897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:12.160182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1135/2268] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:59:12.164621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213c5ce-809e-4a32-a84a-5cbccf6842d0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:12.173976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213c5ce-809e-4a32-a84a-5cbccf6842d0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:12.179554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:12.180338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.180619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.180888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.181623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:12.184114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:12.213583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1134/2269] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:12 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:59:12.217523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:12.227284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07422a14-7a11-4de1-a117-a60f9b3aafa1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:12.291531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:12.292208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.292491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.292793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.293537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:12.295477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:12.318148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1136/2270] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:12 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:12.322182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:12.332761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5138b-2877-4630-a724-0a8aba9ce5f2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:12.337598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:12.338252 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.338531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.338801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.339516 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:12.341090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:12.351208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1135/2271] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:59:12.355155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:12.364992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e70d13-1857-4e28-81a2-019290b12555 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:12.369549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71aab91-871c-492c-878d-38aafb03c694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:12.370199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71aab91-871c-492c-878d-38aafb03c694 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.370486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71aab91-871c-492c-878d-38aafb03c694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.370762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71aab91-871c-492c-878d-38aafb03c694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.372114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71aab91-871c-492c-878d-38aafb03c694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:12.374024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71aab91-871c-492c-878d-38aafb03c694 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:12.385291 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1137/2272] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:12.389735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:12.399958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a81c2-c956-41f1-9f3e-89db05aaf095 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:12.404636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:12.405562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.405850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.406153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] PATH_INFO: `/v3/projects/d66523019fbc451482318e23508844f0/users/1a4eb8764cb94663b0cdcf2adedc4f69/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.421198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d66523019fbc451482318e23508844f0, user_id=1a4eb8764cb94663b0cdcf2adedc4f69, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:12.423851 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:12.437999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1136/2273] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:12 2022] PUT /identity/v3/projects/d66523019fbc451482318e23508844f0/users/1a4eb8764cb94663b0cdcf2adedc4f69/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:12.449056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.450117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.450214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.478285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] MFA Rules not processed for user `1a4eb8764cb94663b0cdcf2adedc4f69`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:12.479497 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:12.484039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:12.552333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1138/2274] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:12.564329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:12.566057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:12.623372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f9159-eb69-423c-8986-1a69be165d02 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:12.627053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:12.627699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:12.627987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:12.628269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:12.636863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:12.638370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:12.647198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1137/2275] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:21.309121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:21.310631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:21.407026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:21.408571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:21.420316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:21.459273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038de73-31fb-4f74-bfef-db9f52c151ad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:21.465503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:21.466174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:21.466457 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:21.466730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:21.468998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:21.484341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:21.486161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:21.501046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1139/2276] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:21.501935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b2cbb-9578-446a-bd28-24b03498fd23 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:21.523177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7083624-1aff-4c1e-a61a-7d196da8e907 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:21.523858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7083624-1aff-4c1e-a61a-7d196da8e907 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:21.524403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7083624-1aff-4c1e-a61a-7d196da8e907 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:21.524688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7083624-1aff-4c1e-a61a-7d196da8e907 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:21.539146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7083624-1aff-4c1e-a61a-7d196da8e907 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:21.541931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7083624-1aff-4c1e-a61a-7d196da8e907 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:21.560361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1138/2277] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:59:21.949525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:22.021589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6181ff6-4089-4bb9-b067-2f2989678c71 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:22.026525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:22.028000 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:22.028280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:22.028556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] PATH_INFO: `/v3/users/1a4eb8764cb94663b0cdcf2adedc4f69` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:22.029775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4eb8764cb94663b0cdcf2adedc4f69)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:22.034294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:22.057493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:22.097060 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4eb8764cb94663b0cdcf2adedc4f69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.106323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4eb8764cb94663b0cdcf2adedc4f69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.119590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4eb8764cb94663b0cdcf2adedc4f69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.120250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4eb8764cb94663b0cdcf2adedc4f69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.120907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4eb8764cb94663b0cdcf2adedc4f69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.133113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1140/2278] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:59:21 2022] DELETE /identity/v3/users/1a4eb8764cb94663b0cdcf2adedc4f69 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:22.137711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:22.138000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:22.138276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:22.155576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:22.156757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:22.227143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1139/2279] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:22.238490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:22.239955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:22.349863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55c796-4e75-4fe4-965c-6a7a672fa6c9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:22.353882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd08719-a4ea-41c0-9327-c80b6fd94f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:22.354527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08719-a4ea-41c0-9327-c80b6fd94f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:22.354804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08719-a4ea-41c0-9327-c80b6fd94f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:22.355099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08719-a4ea-41c0-9327-c80b6fd94f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:22.368755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd08719-a4ea-41c0-9327-c80b6fd94f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:22.371304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd08719-a4ea-41c0-9327-c80b6fd94f73 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:22.388731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1141/2280] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:22.706089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:22.708023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:22.721733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7908b-2a58-4126-89aa-bb6ee43ac629 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:22.729298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:22.730304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:22.730593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:22.730948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] PATH_INFO: `/v3/projects/d66523019fbc451482318e23508844f0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:22.732112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d66523019fbc451482318e23508844f0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:22.734140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:22.783572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd66523019fbc451482318e23508844f0', 'request_id': 'req-6ee5faed-82a9-4d35-8f74-67895d263968'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.789800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd66523019fbc451482318e23508844f0', 'request_id': 'req-6ee5faed-82a9-4d35-8f74-67895d263968'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.795725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd66523019fbc451482318e23508844f0', 'request_id': 'req-6ee5faed-82a9-4d35-8f74-67895d263968'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.813020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] The token cache is being invalidate because project d66523019fbc451482318e23508844f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:59:22.814061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.814726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6ee5faed-82a9-4d35-8f74-67895d263968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:22.816645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1140/2281] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:59:22 2022] DELETE /identity/v3/projects/d66523019fbc451482318e23508844f0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:59:22.822572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e9d2b-8ce3-4a10-9c67-cefaac6a75f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:22.822799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e9d2b-8ce3-4a10-9c67-cefaac6a75f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:22.823081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e9d2b-8ce3-4a10-9c67-cefaac6a75f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:22.845392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5d3e9d2b-8ce3-4a10-9c67-cefaac6a75f0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:22.847098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d3e9d2b-8ce3-4a10-9c67-cefaac6a75f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:22.939507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1142/2282] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:22.941067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f576d6-e4fc-4d30-abc0-914c4f3a2072 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:22.941658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f576d6-e4fc-4d30-abc0-914c4f3a2072 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:22.942041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f576d6-e4fc-4d30-abc0-914c4f3a2072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:22.968167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-68f576d6-e4fc-4d30-abc0-914c4f3a2072 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:22.969552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68f576d6-e4fc-4d30-abc0-914c4f3a2072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:22.981303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1141/2283] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:59:22.984041 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e9d2b-8ce3-4a10-9c67-cefaac6a75f0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:22.996401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e9d2b-8ce3-4a10-9c67-cefaac6a75f0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:23.003178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:23.003863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.004147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.004430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.005198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:59:23.005516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:23.007467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:23.026363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1143/2284] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:59:23.027185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f576d6-e4fc-4d30-abc0-914c4f3a2072 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:23.038271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f576d6-e4fc-4d30-abc0-914c4f3a2072 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:23.044880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:23.045604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.045895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.046157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.046858 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:23.048811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:23.064226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:23.092506 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1142/2285] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:23 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:59:23.098330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:23.114040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd2e0-7d9c-47b2-b405-f4bf139523e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:23.145321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:23.203318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:23.204006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.204818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.204818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.205536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:23.207620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:23.233186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1144/2286] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:23 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:23.237073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:23.248272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d80b7-5bf3-4044-9819-2b64ea6de0f5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:23.253173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:23.253867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.254628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.254910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.255645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:23.257335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:23.267987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1143/2287] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:23.272056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:23.282487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389f4b0-eb70-42e3-8504-25895ac120d3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:23.287172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d9d34-0f28-4f0f-81a7-aff2c98c3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:23.287873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9d34-0f28-4f0f-81a7-aff2c98c3947 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.288176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9d34-0f28-4f0f-81a7-aff2c98c3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.288456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9d34-0f28-4f0f-81a7-aff2c98c3947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.289214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d9d34-0f28-4f0f-81a7-aff2c98c3947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:23.290817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d9d34-0f28-4f0f-81a7-aff2c98c3947 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:23.303304 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1145/2288] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:59:23.307337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:23.317902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c03a39-3948-49a0-96ba-bbf7162aa495 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:23.322477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:23.323353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.323676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.323957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] PATH_INFO: `/v3/projects/4d13c9a0ad1146158525c3398ff4de2a/users/ecacca157e3a43b79659f78e4beb0c5e/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.338333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d13c9a0ad1146158525c3398ff4de2a, user_id=ecacca157e3a43b79659f78e4beb0c5e, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:23.341503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:23.354413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1144/2289] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:23 2022] PUT /identity/v3/projects/4d13c9a0ad1146158525c3398ff4de2a/users/ecacca157e3a43b79659f78e4beb0c5e/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:23.360129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.360594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.360803 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.378278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:23.379601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:23.446597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1146/2290] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:23.456600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:23.458509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:23.516168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045adede-a151-449e-b2e4-0b06c0f3a2a3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:23.519771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a53ef-7511-47e4-8896-40cf59712d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:23.520416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a53ef-7511-47e4-8896-40cf59712d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:23.521317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a53ef-7511-47e4-8896-40cf59712d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:23.521317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a53ef-7511-47e4-8896-40cf59712d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:23.530668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a53ef-7511-47e4-8896-40cf59712d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:23.532101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a53ef-7511-47e4-8896-40cf59712d9c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:23.541631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1145/2291] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:24.527371 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:24.528618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:24.541080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed570-8fd0-4122-b1a7-34fb19056196 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:24.560764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:24.561746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:24.562012 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:24.562289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] PATH_INFO: `/v3/users/91c1d39e9f7e4b11bfe56b967dc3b5d2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:24.563504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c1d39e9f7e4b11bfe56b967dc3b5d2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:24.565537 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:24.624694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c1d39e9f7e4b11bfe56b967dc3b5d2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:24.636391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c1d39e9f7e4b11bfe56b967dc3b5d2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:24.650715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c1d39e9f7e4b11bfe56b967dc3b5d2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:24.651435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c1d39e9f7e4b11bfe56b967dc3b5d2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:24.652076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c1d39e9f7e4b11bfe56b967dc3b5d2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:24.667748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1147/2292] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:59:24 2022] DELETE /identity/v3/users/91c1d39e9f7e4b11bfe56b967dc3b5d2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:24.672789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:24.673378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:24.673378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:24.706497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:24.707707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:24.782452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1146/2293] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:24.794343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:24.796556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:24.820808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:24.880334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2b0dd-369c-4a38-bd0b-cdb12012610e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:24.885287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89a0b-672d-404c-9b90-d7ba71af2ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:24.886416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89a0b-672d-404c-9b90-d7ba71af2ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:24.886672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89a0b-672d-404c-9b90-d7ba71af2ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:24.886930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89a0b-672d-404c-9b90-d7ba71af2ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:24.899403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89a0b-672d-404c-9b90-d7ba71af2ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:24.901022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89a0b-672d-404c-9b90-d7ba71af2ad7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:24.912785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1148/2294] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:59:25.575247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:25.576774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:25.601671 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca2cf4-c63b-4743-824c-0809f59d3fd0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:25.641082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:25.644616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:25.644616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:25.644616 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] PATH_INFO: `/v3/projects/0e0fb36c1dc945419b201c725a0c7bbd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:25.645546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0fb36c1dc945419b201c725a0c7bbd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:25.651300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:25.786261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0fb36c1dc945419b201c725a0c7bbd', 'request_id': 'req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:25.794648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:25.807540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0fb36c1dc945419b201c725a0c7bbd', 'request_id': 'req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:25.825736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0fb36c1dc945419b201c725a0c7bbd', 'request_id': 'req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:25.864911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] The token cache is being invalidate because project 0e0fb36c1dc945419b201c725a0c7bbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:59:25.865444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:25.866774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-064e2d99-c1f5-4e73-997c-b8cf4d6b4985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:25.868967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1147/2295] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:59:25 2022] DELETE /identity/v3/projects/0e0fb36c1dc945419b201c725a0c7bbd => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:25.875704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24bae-9043-41b3-9047-c8a10a619b5c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:25.876249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24bae-9043-41b3-9047-c8a10a619b5c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:25.876448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24bae-9043-41b3-9047-c8a10a619b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:25.907807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3de24bae-9043-41b3-9047-c8a10a619b5c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:25.909044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3de24bae-9043-41b3-9047-c8a10a619b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:26.034675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1149/2296] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:26.043502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b11c77-a3bd-4946-ac93-61fa220409b3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.043900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b11c77-a3bd-4946-ac93-61fa220409b3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.044393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b11c77-a3bd-4946-ac93-61fa220409b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.065224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-87b11c77-a3bd-4946-ac93-61fa220409b3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:26.067558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-87b11c77-a3bd-4946-ac93-61fa220409b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:26.081662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24bae-9043-41b3-9047-c8a10a619b5c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.088427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1148/2297] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:26.097593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24bae-9043-41b3-9047-c8a10a619b5c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:26.103764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:26.104757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.105219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.105496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.106387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:59:26.106697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:26.108587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:26.122608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1150/2298] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:59:26.126862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b11c77-a3bd-4946-ac93-61fa220409b3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.137995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b11c77-a3bd-4946-ac93-61fa220409b3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:26.143005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:26.143662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.143662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.143863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.144527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:26.146513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:26.182318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.182515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1149/2299] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:26 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:26.200607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40d9da-5454-4c39-b65e-e5d348965e4e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:26.303200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:26.304357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.304852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.305394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.306483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:26.310207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:26.336084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1151/2300] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:26 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:59:26.340877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.355964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689f69-2b17-40b2-8338-e16f159351a4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:26.366367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:26.367228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.367505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.367850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.368657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:26.372270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:26.391415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1150/2301] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:26.396484 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.410511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51be6b8-c3d6-4af8-bfa8-0a6c9e6ceb0b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:26.420788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:26.422155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.422685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.423195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.424147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:26.426049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:26.449375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.451031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1152/2302] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:26.462050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae9501-5a14-49a1-8ea4-300656e521ed admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:26.476447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2e7ae-72dc-4e94-87af-0168e40bf4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:26.477702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2e7ae-72dc-4e94-87af-0168e40bf4d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.477992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2e7ae-72dc-4e94-87af-0168e40bf4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.478272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2e7ae-72dc-4e94-87af-0168e40bf4d6 admin admin] PATH_INFO: `/v3/projects/983cd21cc7804d9bbe9590dd6bd34358/users/01d83fab97154876b393d2e639f891aa/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.513320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2e7ae-72dc-4e94-87af-0168e40bf4d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983cd21cc7804d9bbe9590dd6bd34358, user_id=01d83fab97154876b393d2e639f891aa, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:26.516289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2e7ae-72dc-4e94-87af-0168e40bf4d6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:26.547424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.549406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:26.550585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1151/2303] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:26 2022] PUT /identity/v3/projects/983cd21cc7804d9bbe9590dd6bd34358/users/01d83fab97154876b393d2e639f891aa/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:59:26.564696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.564991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.565312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.594714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabad3a-d6be-41aa-a0e5-3eab712491f3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:26.611388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] MFA Rules not processed for user `01d83fab97154876b393d2e639f891aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:26.612657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:26.703413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:26.807289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:26.808503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1152/2304] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:26.818610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:26.818912 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:26.819202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] PATH_INFO: `/v3/users/36e6babf99e24bcdb233ec89a27771f2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:26.823160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:26.823270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e6babf99e24bcdb233ec89a27771f2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:26.825467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:26.832346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:26.953735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e6babf99e24bcdb233ec89a27771f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:26.986756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e6babf99e24bcdb233ec89a27771f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:26.999857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeead381-a377-468b-84d6-9db2618195d9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:27.013796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:27.014473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:27.014762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:27.015041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:27.030376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e6babf99e24bcdb233ec89a27771f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.031552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e6babf99e24bcdb233ec89a27771f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.034946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6fe6f361-9e11-4580-bf76-42c4d58f0636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e6babf99e24bcdb233ec89a27771f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.051098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:27.054740 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:27.074997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1153/2305] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:59:26 2022] DELETE /identity/v3/users/36e6babf99e24bcdb233ec89a27771f2 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:59:27.085799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:27.086092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:27.086375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:27.093847 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1153/2306] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:59:27.114709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:27.116632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:27.134803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:27.151967 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57d0f3-11c8-40b5-86ba-008b3acf2f3a service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:27.166991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:27.314193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1154/2307] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:27.328321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:27.331327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:27.332782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:27.333077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] PATH_INFO: `/v3/users/01d83fab97154876b393d2e639f891aa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:27.335090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d83fab97154876b393d2e639f891aa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:27.338354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:27.377316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:27.379064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:27.473706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d83fab97154876b393d2e639f891aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.514157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d83fab97154876b393d2e639f891aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.514955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d83fab97154876b393d2e639f891aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.515596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d83fab97154876b393d2e639f891aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.525375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:27.541107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf0694c7-adda-46cf-92a9-ed3989cf1909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d83fab97154876b393d2e639f891aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:27.562319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13316c0d-63a5-460c-bcb9-e2f7488e644f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:27.586744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:27.588088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:27.588088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:27.590018 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:27.592997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:27.604395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1154/2308] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 13:59:27 2022] DELETE /identity/v3/users/01d83fab97154876b393d2e639f891aa => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:27.609294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:27.609957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:27.610576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:27.663742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:27.664926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:27.678468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:27.681058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:27.696133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1155/2309] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:27.776051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1155/2310] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:27.807375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:27.813138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:27.827740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d7c10-ec90-4228-98de-d2b481b7bc03 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:27.934805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07b510-014d-4d14-9bdb-6ac50eae2232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:27.935809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b510-014d-4d14-9bdb-6ac50eae2232 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:27.936249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b510-014d-4d14-9bdb-6ac50eae2232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:27.936675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b510-014d-4d14-9bdb-6ac50eae2232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:27.967146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07b510-014d-4d14-9bdb-6ac50eae2232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:27.975571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07b510-014d-4d14-9bdb-6ac50eae2232 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:28.010835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1156/2311] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:28.464033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:28.466900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:28.486946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba4224-5529-42c2-b79f-ef90e09fbccc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:28.491788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:28.492794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:28.493092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:28.493610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] PATH_INFO: `/v3/projects/5ad7e1c6adb0483397b3b2de2ac60ec8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:28.495371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad7e1c6adb0483397b3b2de2ac60ec8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:28.497957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:28.533378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:28.602963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad7e1c6adb0483397b3b2de2ac60ec8', 'request_id': 'req-d301ca64-5e10-4022-b45f-5ab232bf83a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:28.621120 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad7e1c6adb0483397b3b2de2ac60ec8', 'request_id': 'req-d301ca64-5e10-4022-b45f-5ab232bf83a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:28.640953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad7e1c6adb0483397b3b2de2ac60ec8', 'request_id': 'req-d301ca64-5e10-4022-b45f-5ab232bf83a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:28.676810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] The token cache is being invalidate because project 5ad7e1c6adb0483397b3b2de2ac60ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:59:28.677106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:28.678658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d301ca64-5e10-4022-b45f-5ab232bf83a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:28.680580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1156/2312] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:59:28 2022] DELETE /identity/v3/projects/5ad7e1c6adb0483397b3b2de2ac60ec8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:28.686624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e49ecb-2d52-4018-98e9-08a3eaa7e0fc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:28.686917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e49ecb-2d52-4018-98e9-08a3eaa7e0fc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:28.687193 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e49ecb-2d52-4018-98e9-08a3eaa7e0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:28.721656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d2e49ecb-2d52-4018-98e9-08a3eaa7e0fc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:28.726376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d2e49ecb-2d52-4018-98e9-08a3eaa7e0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:28.801422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1157/2313] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:28.810805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b3736-1c58-473b-bc6f-292e4046548a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:28.811100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b3736-1c58-473b-bc6f-292e4046548a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:28.811562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b3736-1c58-473b-bc6f-292e4046548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:28.847009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9f5b3736-1c58-473b-bc6f-292e4046548a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:28.850346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9f5b3736-1c58-473b-bc6f-292e4046548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:28.872202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1157/2314] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:28.878975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e49ecb-2d52-4018-98e9-08a3eaa7e0fc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:28.891158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e49ecb-2d52-4018-98e9-08a3eaa7e0fc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:28.900268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:28.902407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:28.902834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:28.903865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:28.904667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:59:28.906324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:28.913784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:28.947249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1158/2315] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:59:28.949332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b3736-1c58-473b-bc6f-292e4046548a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:28.969644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b3736-1c58-473b-bc6f-292e4046548a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:28.982589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:28.983457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:28.983736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:28.984009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:28.984887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:28.988269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:28.999342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.004878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:29.027789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de0cbc-8fc3-4ccb-91e9-4f015588a376 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.059110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1158/2316] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:28 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:59:29.063631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.067375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.068517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.069294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.070205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] PATH_INFO: `/v3/projects/983cd21cc7804d9bbe9590dd6bd34358` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.074247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983cd21cc7804d9bbe9590dd6bd34358)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.076848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.163500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983cd21cc7804d9bbe9590dd6bd34358', 'request_id': 'req-fd0aca40-af61-4e84-8752-2782ca5e6740'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:29.164997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb55e5-dbe9-4b44-bd78-2339efe6b4fb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.173575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983cd21cc7804d9bbe9590dd6bd34358', 'request_id': 'req-fd0aca40-af61-4e84-8752-2782ca5e6740'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:29.181908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983cd21cc7804d9bbe9590dd6bd34358', 'request_id': 'req-fd0aca40-af61-4e84-8752-2782ca5e6740'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:29.194666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:29.206607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] The token cache is being invalidate because project 983cd21cc7804d9bbe9590dd6bd34358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:59:29.207422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:29.213385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fd0aca40-af61-4e84-8752-2782ca5e6740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:59:29.216170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1159/2317] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 13:59:28 2022] DELETE /identity/v3/projects/983cd21cc7804d9bbe9590dd6bd34358 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:29.220814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138ce0-7c39-448c-8d9c-56fac279822b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.221283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138ce0-7c39-448c-8d9c-56fac279822b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.221629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138ce0-7c39-448c-8d9c-56fac279822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.243565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d8138ce0-7c39-448c-8d9c-56fac279822b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:29.244901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8138ce0-7c39-448c-8d9c-56fac279822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:29.270445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.271162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.271449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.271730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.276337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.278458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.316690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1159/2318] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/users => generated 290 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:29.316690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.317244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:29.335196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45482df4-786d-4972-842d-3d3ce59e2e26 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.341783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.342933 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.343035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.343403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.344243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.346367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.367212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1160/2319] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:29.371087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.377192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1160/2320] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:59:29.381402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01245a-0b60-402f-898d-983f33f5cfed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.381804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01245a-0b60-402f-898d-983f33f5cfed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.382920 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01245a-0b60-402f-898d-983f33f5cfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.383556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9755572-c03d-4ee4-be16-3da8eec91735 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.388745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.389643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.390077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.390579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.391607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.393267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.395911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:29.407096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e01245a-0b60-402f-898d-983f33f5cfed None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:29.409500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e01245a-0b60-402f-898d-983f33f5cfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:29.411135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1161/2321] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:29.412022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.423517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1161/2322] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:59:29.430138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01245a-0b60-402f-898d-983f33f5cfed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.430647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac902-9e6a-4637-8c4c-95970ea0f595 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.436908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580f8fd-02c5-4d42-809d-e6358e0ea079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.437816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580f8fd-02c5-4d42-809d-e6358e0ea079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.438095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580f8fd-02c5-4d42-809d-e6358e0ea079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.438382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580f8fd-02c5-4d42-809d-e6358e0ea079 admin admin] PATH_INFO: `/v3/projects/ed15631acafd41528b96f2ed12c19e5b/users/5068f06b1a8e47b88ba3294169545aa3/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.445015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01245a-0b60-402f-898d-983f33f5cfed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.451378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.452485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.452963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.453561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.454800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:59:29.455438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.457113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580f8fd-02c5-4d42-809d-e6358e0ea079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed15631acafd41528b96f2ed12c19e5b, user_id=5068f06b1a8e47b88ba3294169545aa3, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.457522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.460107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580f8fd-02c5-4d42-809d-e6358e0ea079 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.473132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1162/2323] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 13:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:59:29.477320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.490328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8249a-b7e4-438d-b6b1-5c7db2b0ac59 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.495605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.496464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.497061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.497369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.498551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.501712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.506739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1162/2324] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:29 2022] PUT /identity/v3/projects/ed15631acafd41528b96f2ed12c19e5b/users/5068f06b1a8e47b88ba3294169545aa3/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:29.511532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.512014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.512458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.551224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:29.551969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1163/2325] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:29.553193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:29.555580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.680745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1163/2326] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:59:29.687738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce144-2db3-4b6c-91ae-7173a6e8ab4a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.698828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.700693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:29.706763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.708800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.710287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.712790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.713821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.716833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.760305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1164/2327] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:59:29.770560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.788718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e45865-8ca4-4da8-8a0c-47a2e5f90184 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.794632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.795638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.796026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.796451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.797429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.799370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.817277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1165/2328] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:29.817484 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fbd57-6581-4d4a-b182-ae60f3e1f23a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.822240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.824390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb2950-eed2-47ce-bb54-e3acc25caff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.826164 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb2950-eed2-47ce-bb54-e3acc25caff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.826615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb2950-eed2-47ce-bb54-e3acc25caff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.827041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb2950-eed2-47ce-bb54-e3acc25caff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.840929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b002091-a317-4fb1-a492-cbe160fc7912 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.845672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.846454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.846749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.847031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.847917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.849419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.850519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb2950-eed2-47ce-bb54-e3acc25caff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.851976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb2950-eed2-47ce-bb54-e3acc25caff9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.864302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1164/2329] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:29.865425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.865733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.866013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.878087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1166/2330] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:29.878990 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:29.892809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43bb27-8601-4201-8a6b-01ebd2cf8a49 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:29.899255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907045bf-c845-4ade-bdc8-ca9b1c4236aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:29.900161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907045bf-c845-4ade-bdc8-ca9b1c4236aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.900449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907045bf-c845-4ade-bdc8-ca9b1c4236aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.900728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907045bf-c845-4ade-bdc8-ca9b1c4236aa admin admin] PATH_INFO: `/v3/projects/08ed6980f1214f6789f94f34a4c68e6e/users/0e7ef003bc5642feadfa1f9f631ef903/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.908026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] MFA Rules not processed for user `ecacca157e3a43b79659f78e4beb0c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:29.909735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:29.932396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907045bf-c845-4ade-bdc8-ca9b1c4236aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ed6980f1214f6789f94f34a4c68e6e, user_id=0e7ef003bc5642feadfa1f9f631ef903, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:29.935417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907045bf-c845-4ade-bdc8-ca9b1c4236aa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:29.955043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1167/2331] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:29 2022] PUT /identity/v3/projects/08ed6980f1214f6789f94f34a4c68e6e/users/0e7ef003bc5642feadfa1f9f631ef903/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:29.960887 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:29.961295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:29.961660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:29.970522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:29.995388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:29.997728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:30.043459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1165/2332] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:30.047932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:30.049444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:30.065325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f619b-a881-46e3-a91b-15588dc20ae3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:30.069427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:30.108658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1168/2333] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:30.117576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:30.119322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:30.160558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:30.161472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:30.161757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:30.162032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:30.163731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:30.166511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:30.202832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2ddb-5863-44fc-b0d0-c8955c796d9c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:30.212634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:30.213575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:30.214600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:30.214983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:30.227701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1166/2334] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:30 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:30.228999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:30.230693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:30.232558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:30.252397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1169/2335] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:59:30.320791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70874559-8f00-43ba-8008-8dcdfba41e1e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:30.326151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:30.326833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:30.327842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:30.327842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:30.328070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:30.330528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:30.357203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1167/2336] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:30 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:30.359567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:30.372288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71e8a-fc5f-4257-bdc8-4a2d8b269dc9 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:30.379650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:30.381485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:30.381993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:30.382473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:30.383319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:30.385954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:30.405242 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1170/2337] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:30.406836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:30.417372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c8c4-827d-476d-be96-685a843c04d1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:30.422476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:30.423371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:30.423653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:30.423934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] PATH_INFO: `/v3/projects/a5028cfaabd84d63a6ff2cb33b0133e4/users/e757da0c768147adb41da77866ed35c9/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:30.446282 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5028cfaabd84d63a6ff2cb33b0133e4, user_id=e757da0c768147adb41da77866ed35c9, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:30.450043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:30.470714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1168/2338] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:30 2022] PUT /identity/v3/projects/a5028cfaabd84d63a6ff2cb33b0133e4/users/e757da0c768147adb41da77866ed35c9/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:30.472228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:30.488648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc062c8-be7a-42e1-b40b-9c9b465ecd37 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:30.570096 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533164f-13b7-4dc8-83da-95beb5b279b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:30.570434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533164f-13b7-4dc8-83da-95beb5b279b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:30.570639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533164f-13b7-4dc8-83da-95beb5b279b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:30.578252 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533164f-13b7-4dc8-83da-95beb5b279b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:30.579045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533164f-13b7-4dc8-83da-95beb5b279b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:30.580875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533164f-13b7-4dc8-83da-95beb5b279b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:30.592223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1171/2339] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:30.596183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:30.608377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e910d89-4bbc-4cc4-a5ce-df760d49920e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:30.614281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e61c4d-3e4b-4703-998e-bd433b1eee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:30.615181 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e61c4d-3e4b-4703-998e-bd433b1eee8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:30.615462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e61c4d-3e4b-4703-998e-bd433b1eee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:30.615765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e61c4d-3e4b-4703-998e-bd433b1eee8d admin admin] PATH_INFO: `/v3/projects/a5028cfaabd84d63a6ff2cb33b0133e4/users/e757da0c768147adb41da77866ed35c9/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:30.617831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e61c4d-3e4b-4703-998e-bd433b1eee8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5028cfaabd84d63a6ff2cb33b0133e4, user_id=e757da0c768147adb41da77866ed35c9, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:30.621079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e61c4d-3e4b-4703-998e-bd433b1eee8d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:30.633180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1169/2340] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:30 2022] PUT /identity/v3/projects/a5028cfaabd84d63a6ff2cb33b0133e4/users/e757da0c768147adb41da77866ed35c9/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:36.947125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:36.948204 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:36.949247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:37.043203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] MFA Rules not processed for user `2eb759cf823b45d2aa35fdcacc7ac3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:37.044619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:37.094481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:37.187376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1172/2341] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:38.875239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338dd2d-bb1e-4381-a148-df2ce4f4ff6f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:38.875724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338dd2d-bb1e-4381-a148-df2ce4f4ff6f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:38.876000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338dd2d-bb1e-4381-a148-df2ce4f4ff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:38.938280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0338dd2d-bb1e-4381-a148-df2ce4f4ff6f None None] MFA Rules not processed for user `e757da0c768147adb41da77866ed35c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:38.939639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0338dd2d-bb1e-4381-a148-df2ce4f4ff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:38.991381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:39.072885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1170/2342] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:39.347428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:39.348770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:39.854126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2f0d8-451a-4f00-ae6a-551b7c9ff778 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:39.873752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f792fd-ab51-44a5-86a8-2fe10672059b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:39.877054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f792fd-ab51-44a5-86a8-2fe10672059b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:39.879833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f792fd-ab51-44a5-86a8-2fe10672059b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:39.880447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f792fd-ab51-44a5-86a8-2fe10672059b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:40.056303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f792fd-ab51-44a5-86a8-2fe10672059b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:40.060270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f792fd-ab51-44a5-86a8-2fe10672059b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:40.089265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1173/2343] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:42.206949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597abf-2555-411f-b8ff-dceb15427207 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:42.207984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597abf-2555-411f-b8ff-dceb15427207 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:42.208425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597abf-2555-411f-b8ff-dceb15427207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:42.229374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ae597abf-2555-411f-b8ff-dceb15427207 None None] MFA Rules not processed for user `ecacca157e3a43b79659f78e4beb0c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:42.230805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae597abf-2555-411f-b8ff-dceb15427207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:42.273209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1171/2344] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:42.765044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ad894-c151-48f0-972f-2a92318ead3b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:42.765551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ad894-c151-48f0-972f-2a92318ead3b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:42.765721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ad894-c151-48f0-972f-2a92318ead3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:42.811632 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-361ad894-c151-48f0-972f-2a92318ead3b None None] MFA Rules not processed for user `0e7ef003bc5642feadfa1f9f631ef903`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:42.813056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-361ad894-c151-48f0-972f-2a92318ead3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:42.903200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:42.970871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1174/2345] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:42.972517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597abf-2555-411f-b8ff-dceb15427207 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:43.128469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597abf-2555-411f-b8ff-dceb15427207 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:43.172861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:43.174146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:43.175063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:43.175574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:43.180748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:43.186841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:43.246793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1172/2346] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:42 2022] POST /identity/v3/projects => generated 383 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:59:43.251050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ad894-c151-48f0-972f-2a92318ead3b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:43.269031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ad894-c151-48f0-972f-2a92318ead3b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:43.366061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:43.366907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:43.367277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:43.367630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:43.368348 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:43.370669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:43.409992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1175/2347] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:43 2022] POST /identity/v3/users => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:43.412126 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:43.440965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d10bf4-7ec4-428e-95cd-9dc5b20eaa9a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:43.453408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:43.454636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:43.454930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:43.455277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:43.456258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:43.458422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:43.493055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1173/2348] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:59:43.497505 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:43.517976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c9d5f-25f5-40a1-a314-4e3c6f339c06 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:43.536727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:43.546674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:43.550468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:43.551865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] PATH_INFO: `/v3/projects/60a6f4f6d3854690befed268a5911a3b/users/e530500987d3441ba10c02c94c92098c/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:43.587720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a6f4f6d3854690befed268a5911a3b, user_id=e530500987d3441ba10c02c94c92098c, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:43.594448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:43.624381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1176/2349] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:43 2022] PUT /identity/v3/projects/60a6f4f6d3854690befed268a5911a3b/users/e530500987d3441ba10c02c94c92098c/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:43.625962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:43.655012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159cfae-92cd-4e03-a58f-c146b383b0f5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:43.817892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e17f29-c7ed-4bdf-8f5f-3480c2908db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:43.818686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e17f29-c7ed-4bdf-8f5f-3480c2908db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:43.819019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e17f29-c7ed-4bdf-8f5f-3480c2908db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:43.819396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e17f29-c7ed-4bdf-8f5f-3480c2908db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:43.820200 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e17f29-c7ed-4bdf-8f5f-3480c2908db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:43.822661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e17f29-c7ed-4bdf-8f5f-3480c2908db7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:43.863318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:43.871528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1174/2350] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:43.888842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401f27-80b3-4c54-a0e8-352758e9369f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:43.894415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:43.895962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:43.896455 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:43.897051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] PATH_INFO: `/v3/projects/60a6f4f6d3854690befed268a5911a3b/users/e530500987d3441ba10c02c94c92098c/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:43.899877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a6f4f6d3854690befed268a5911a3b, user_id=e530500987d3441ba10c02c94c92098c, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:43.903574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:43.926186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1177/2351] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:43 2022] PUT /identity/v3/projects/60a6f4f6d3854690befed268a5911a3b/users/e530500987d3441ba10c02c94c92098c/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:44.423161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1789d6-7e43-4862-a147-92d4b8cbc0a3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:44.423703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1789d6-7e43-4862-a147-92d4b8cbc0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:44.423703 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1789d6-7e43-4862-a147-92d4b8cbc0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:44.463059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4e1789d6-7e43-4862-a147-92d4b8cbc0a3 None None] MFA Rules not processed for user `5068f06b1a8e47b88ba3294169545aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:44.465580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4e1789d6-7e43-4862-a147-92d4b8cbc0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:44.491321 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:44.553936 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1175/2352] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:44.558890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:44.759427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62041eb9-55ec-403c-979d-0b2b1a103e5e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:44.769188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:44.770635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:44.770635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:44.770635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:44.771903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:44.774495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:44.824682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1789d6-7e43-4862-a147-92d4b8cbc0a3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:44.837570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1178/2353] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 13:59:44 2022] POST /identity/v3/projects => generated 379 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:44.846370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1789d6-7e43-4862-a147-92d4b8cbc0a3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:44.946676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:44.947463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:44.948474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:44.948822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:44.949630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:44.955393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:45.004160 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1176/2354] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 13:59:44 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:59:45.008336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:45.028692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b8ad-4620-4912-badd-0bf4fb0b4a0e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:45.037083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:45.039220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:45.039655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:45.040733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:45.043753 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:45.047733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:45.089323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1179/2355] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:59:45.094413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:45.111707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1d36f-724d-44b3-b86a-62199bb79699 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:45.125295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:45.126212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:45.126501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:45.126783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] PATH_INFO: `/v3/projects/e23df072c3ad4ce6b25f813c47cb1d1f/users/376e56dc18f14fabb7265c74a152dc69/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:45.175677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23df072c3ad4ce6b25f813c47cb1d1f, user_id=376e56dc18f14fabb7265c74a152dc69, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:45.188370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:45.239253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1177/2356] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:45 2022] PUT /identity/v3/projects/e23df072c3ad4ce6b25f813c47cb1d1f/users/376e56dc18f14fabb7265c74a152dc69/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:59:45.245192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:45.250866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:45.267383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149640-3927-4861-9b15-93c563f81957 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:45.368695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20168dca-9fc5-486e-bc46-fe84ab92ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:45.370361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20168dca-9fc5-486e-bc46-fe84ab92ffc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:45.370874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20168dca-9fc5-486e-bc46-fe84ab92ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:45.371498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20168dca-9fc5-486e-bc46-fe84ab92ffc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:45.372517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20168dca-9fc5-486e-bc46-fe84ab92ffc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:45.374592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20168dca-9fc5-486e-bc46-fe84ab92ffc8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:45.393799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1180/2357] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 13:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:59:45.397874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:45.412140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15000-2346-4464-99aa-416b660b85b9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:45.418099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:45.419259 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:45.419758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:45.420323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] PATH_INFO: `/v3/projects/e23df072c3ad4ce6b25f813c47cb1d1f/users/376e56dc18f14fabb7265c74a152dc69/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:45.422669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23df072c3ad4ce6b25f813c47cb1d1f, user_id=376e56dc18f14fabb7265c74a152dc69, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:45.426025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:45.446448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1178/2358] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 13:59:45 2022] PUT /identity/v3/projects/e23df072c3ad4ce6b25f813c47cb1d1f/users/376e56dc18f14fabb7265c74a152dc69/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:59:52.680765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b69409-5dc0-4cc7-8556-286ea5a27937 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:52.681362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b69409-5dc0-4cc7-8556-286ea5a27937 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:52.681588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b69409-5dc0-4cc7-8556-286ea5a27937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:52.742260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-39b69409-5dc0-4cc7-8556-286ea5a27937 None None] MFA Rules not processed for user `e530500987d3441ba10c02c94c92098c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:52.743650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39b69409-5dc0-4cc7-8556-286ea5a27937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:52.888868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1181/2359] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:52.905009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:52.908691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:53.104597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fef3a7-0943-4e31-a3a6-0964f35039a2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:53.108729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:53.109448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:53.109732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:53.110013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:53.120195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:53.244032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:53.245914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:53.270984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1179/2360] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:53.504621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f46df-fb9c-4baf-a0d7-59c554b65cf8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:53.505248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f46df-fb9c-4baf-a0d7-59c554b65cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:53.505692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f46df-fb9c-4baf-a0d7-59c554b65cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:53.581928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-156f46df-fb9c-4baf-a0d7-59c554b65cf8 None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:53.583321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-156f46df-fb9c-4baf-a0d7-59c554b65cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:53.665822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:53.677701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1182/2361] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 13:59:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:53.791610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:53.793071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:53.941752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e3cc7-3087-48b3-8f90-95b0fea2c004 service glance] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:53.948438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:53.949670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:53.950052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:53.950461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:53.960159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:53.962459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:53.973425 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1180/2362] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 13:59:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:59:54.792805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8937-ea16-416a-a15e-25b7cda8001d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:54.793434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8937-ea16-416a-a15e-25b7cda8001d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:54.793434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8937-ea16-416a-a15e-25b7cda8001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:54.854231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dccc8937-ea16-416a-a15e-25b7cda8001d None None] MFA Rules not processed for user `376e56dc18f14fabb7265c74a152dc69`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:59:54.856164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dccc8937-ea16-416a-a15e-25b7cda8001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:55.139298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1183/2363] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 13:59:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:59:55.161458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:55.162918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:55.381094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb899b-2ab3-44be-8893-6972c3453eb3 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:55.389986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fed46c-d788-4302-9a61-347f5d9796ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:55.395198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed46c-d788-4302-9a61-347f5d9796ba service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:55.395495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed46c-d788-4302-9a61-347f5d9796ba service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:55.396099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed46c-d788-4302-9a61-347f5d9796ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:55.421951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:55.482666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fed46c-d788-4302-9a61-347f5d9796ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:55.484319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fed46c-d788-4302-9a61-347f5d9796ba service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:55.515999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1181/2364] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 13:59:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:00:05.120885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=36 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:05.496477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=20 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:08.686893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b91d3-440a-4c98-95cd-9448bdc5b32c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:08.687456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b91d3-440a-4c98-95cd-9448bdc5b32c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:08.687456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b91d3-440a-4c98-95cd-9448bdc5b32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:08.710595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-822b91d3-440a-4c98-95cd-9448bdc5b32c None None] MFA Rules not processed for user `0e7ef003bc5642feadfa1f9f631ef903`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:08.711865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-822b91d3-440a-4c98-95cd-9448bdc5b32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:08.721253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1184/2365] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:00:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:00:11.275752 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a422f27-9d76-4613-ae4b-fe85d39745b0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:11.276308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a422f27-9d76-4613-ae4b-fe85d39745b0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:11.276308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a422f27-9d76-4613-ae4b-fe85d39745b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:11.306135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4a422f27-9d76-4613-ae4b-fe85d39745b0 None None] MFA Rules not processed for user `592975b43454407abde6c7cab2504982`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:11.307371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4a422f27-9d76-4613-ae4b-fe85d39745b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:11.392661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1182/2366] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:12.051976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c719-2418-4afe-9948-e56668b92b66 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:12.052491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c719-2418-4afe-9948-e56668b92b66 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:12.052603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c719-2418-4afe-9948-e56668b92b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:12.071804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3f99c719-2418-4afe-9948-e56668b92b66 None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:12.072739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3f99c719-2418-4afe-9948-e56668b92b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:12.080859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1185/2367] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 14:00:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:12.101269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a422f27-9d76-4613-ae4b-fe85d39745b0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:12.115859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a422f27-9d76-4613-ae4b-fe85d39745b0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:12.121015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:12.121792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:12.122155 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:12.122530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:12.131369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:12.132906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:12.141606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1183/2368] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:00:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:00:21.717531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713cedc-1f58-4ed3-9ca3-c11ca03a9756 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:21.718548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713cedc-1f58-4ed3-9ca3-c11ca03a9756 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:21.718981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713cedc-1f58-4ed3-9ca3-c11ca03a9756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:21.738937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a713cedc-1f58-4ed3-9ca3-c11ca03a9756 None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:21.740477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a713cedc-1f58-4ed3-9ca3-c11ca03a9756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:21.753467 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1186/2369] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 14:00:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:21.778717 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:21.794526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9284b-76d6-41a5-ba82-cef6e9790720 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:21.802260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4a260-2014-4d76-892b-cb8e9363f740 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:21.803202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a260-2014-4d76-892b-cb8e9363f740 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:21.803760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a260-2014-4d76-892b-cb8e9363f740 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:21.804355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a260-2014-4d76-892b-cb8e9363f740 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:21.816487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4a260-2014-4d76-892b-cb8e9363f740 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:21.818352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4a260-2014-4d76-892b-cb8e9363f740 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:21.828483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1184/2370] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:00:31.732321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=9 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:31.824038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=37 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:34.451160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff5c85-7e9c-4dac-9f32-965282bc51ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:34.451691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff5c85-7e9c-4dac-9f32-965282bc51ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:34.451823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff5c85-7e9c-4dac-9f32-965282bc51ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:34.480017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-24ff5c85-7e9c-4dac-9f32-965282bc51ab None None] MFA Rules not processed for user `5068f06b1a8e47b88ba3294169545aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:34.482483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-24ff5c85-7e9c-4dac-9f32-965282bc51ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:34.490752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1187/2371] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:00:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:41.028173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:41.028699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:41.028877 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:41.047488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:41.049481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:41.059732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1185/2372] 10.222.0.91 () {58 vars in 967 bytes} [Thu May 26 14:00:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:41.070947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff5c85-7e9c-4dac-9f32-965282bc51ab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:41.084257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff5c85-7e9c-4dac-9f32-965282bc51ab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:41.099068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:41.099745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:41.100035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:41.100313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:41.109338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:41.110967 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:41.119647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1188/2373] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:00:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:00:43.409378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:43.411975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:43.487581 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46beffa-7784-4b22-89cb-e57cb95a441e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:43.492156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:43.493787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:43.494191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:43.494476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] PATH_INFO: `/v3/users/2eb759cf823b45d2aa35fdcacc7ac3e8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:43.495756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb759cf823b45d2aa35fdcacc7ac3e8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:43.497797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:43.567930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb759cf823b45d2aa35fdcacc7ac3e8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.580574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb759cf823b45d2aa35fdcacc7ac3e8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.581351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb759cf823b45d2aa35fdcacc7ac3e8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.581960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb759cf823b45d2aa35fdcacc7ac3e8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.590642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-90b59b3d-29be-4c59-9488-0808d554c8cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb759cf823b45d2aa35fdcacc7ac3e8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.604539 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1186/2374] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:00:43 2022] DELETE /identity/v3/users/2eb759cf823b45d2aa35fdcacc7ac3e8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:00:43.797865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:43.810417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aa961-9d02-478b-b78d-263b8a7cca9b service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:43.877084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:43.877980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:43.878267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:43.878541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] PATH_INFO: `/v3/projects/12082647641d4ece974c66384e3bf169` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:43.879738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12082647641d4ece974c66384e3bf169)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:43.881824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:43.935719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12082647641d4ece974c66384e3bf169', 'request_id': 'req-4a64f2bb-4832-440a-b818-628960d83ab9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.946789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12082647641d4ece974c66384e3bf169', 'request_id': 'req-4a64f2bb-4832-440a-b818-628960d83ab9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.956140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12082647641d4ece974c66384e3bf169', 'request_id': 'req-4a64f2bb-4832-440a-b818-628960d83ab9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.970807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] The token cache is being invalidate because project 12082647641d4ece974c66384e3bf169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:00:43.971166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.972208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a64f2bb-4832-440a-b818-628960d83ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:00:43.974133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1189/2375] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:00:43 2022] DELETE /identity/v3/projects/12082647641d4ece974c66384e3bf169 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:00:43.978795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dcdb9-c8c4-42da-a35b-ec0ee5c7b018 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:43.979089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dcdb9-c8c4-42da-a35b-ec0ee5c7b018 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:43.979372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dcdb9-c8c4-42da-a35b-ec0ee5c7b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:43.999852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0d3dcdb9-c8c4-42da-a35b-ec0ee5c7b018 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:44.001067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0d3dcdb9-c8c4-42da-a35b-ec0ee5c7b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:44.071959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1187/2376] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:00:44.079618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1688-f08e-4584-8592-5c8f5b7c93c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.079956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1688-f08e-4584-8592-5c8f5b7c93c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.080233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1688-f08e-4584-8592-5c8f5b7c93c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.100952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c44e1688-f08e-4584-8592-5c8f5b7c93c3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:44.102482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c44e1688-f08e-4584-8592-5c8f5b7c93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:44.111377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1190/2377] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:44.115605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dcdb9-c8c4-42da-a35b-ec0ee5c7b018 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:44.125683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dcdb9-c8c4-42da-a35b-ec0ee5c7b018 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:44.131140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:44.131813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.132098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.132375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.133261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:00:44.134170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:44.136062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:44.145618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:44.154035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1188/2378] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:00:44.155055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e1688-f08e-4584-8592-5c8f5b7c93c3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:44.164799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e1688-f08e-4584-8592-5c8f5b7c93c3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:44.169507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:44.170169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.170461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.170768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.171475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:44.173409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:44.200052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1191/2379] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:00:44 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:00:44.204150 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:44.215229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8559503-0fa5-4da2-942e-23f23ed487d0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:44.272919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:44.273629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.273913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.274820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.275523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:44.277496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:44.299192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1189/2380] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:00:44 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:00:44.303356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:44.315114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05669-19a0-40cc-b20e-4a451d2f0d69 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:44.319892 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:44.320562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.320849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.321707 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.322494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:44.324059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:44.335921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1192/2381] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:00:44.340182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:44.350404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cf8e9-8bca-4d4c-900c-bb9e6182ed2f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:44.355226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceb328-9915-4b53-85cc-85472cf31926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:44.355912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb328-9915-4b53-85cc-85472cf31926 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.356195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb328-9915-4b53-85cc-85472cf31926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.356474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb328-9915-4b53-85cc-85472cf31926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.357260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ceb328-9915-4b53-85cc-85472cf31926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:44.359718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ceb328-9915-4b53-85cc-85472cf31926 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:44.375466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1190/2382] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:00:44.380015 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:44.392975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88768dd0-e389-48eb-8680-c6372aaf168a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:44.398078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:44.399010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.399282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.399553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] PATH_INFO: `/v3/projects/2e57d6db85a644e38660e2d0e3016a94/users/d6d5ab170e964086a00f2fdaade4a25a/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.417333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e57d6db85a644e38660e2d0e3016a94, user_id=d6d5ab170e964086a00f2fdaade4a25a, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:44.420153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:44.438240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1193/2383] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:00:44 2022] PUT /identity/v3/projects/2e57d6db85a644e38660e2d0e3016a94/users/d6d5ab170e964086a00f2fdaade4a25a/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:00:44.450571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca67945-d61b-45bd-acc1-7a61bd1a782a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.451077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca67945-d61b-45bd-acc1-7a61bd1a782a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.451199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca67945-d61b-45bd-acc1-7a61bd1a782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.476950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9ca67945-d61b-45bd-acc1-7a61bd1a782a None None] MFA Rules not processed for user `d6d5ab170e964086a00f2fdaade4a25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:44.478260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9ca67945-d61b-45bd-acc1-7a61bd1a782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:44.546105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1191/2384] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:00:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:44.557897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:44.559596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:44.571865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:44.651974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68f961-7c43-411e-8c0b-5f97d5361640 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:44.662572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:44.663759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:44.664054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:44.664846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:44.682874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:44.684419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:44.697930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1194/2385] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:00:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:00:45.905937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:45.906494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:45.907106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:45.932757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:45.941405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:46.045229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1192/2386] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 14:00:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:00:46.067223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:46.068801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:46.207271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8967d6f-19e5-4eda-bbb0-9a86b15f084d service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:46.215827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:46.217485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:46.217821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:46.218058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:46.229955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:46.231925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:46.243646 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1195/2387] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:00:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:00:56.040434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:56.238071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=55 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:59.818715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:59.820080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:59.878472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f583ac-6e41-4f12-99fa-484b501e7239 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:59.882177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:59.882991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:59.883275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:59.883555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:59.885575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:59.951281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:59.952721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:59.962586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1193/2388] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:00:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:03.344213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:03.345870 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:03.357818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ffa04-4b70-4226-9caa-213fa51b5cc4 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:03.361496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:03.362143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:03.362430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:03.362712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:03.364490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:03.438468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:03.439927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:03.452142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1196/2389] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:03.914500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:03.924265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c371e-d371-46d4-af65-9abaa6c64bf8 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:03.929137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:03.930174 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:03.930686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:03.931235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:03.933334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:04.014859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:04.016232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:04.026715 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1194/2390] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:05.051939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:05.062585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cc7a7-d8e6-4339-aec5-ed0d5a6f42ed service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:05.066289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f184a31-2b1c-43da-8b1f-5219554b130e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:05.066947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f184a31-2b1c-43da-8b1f-5219554b130e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:05.067233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f184a31-2b1c-43da-8b1f-5219554b130e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:05.067512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f184a31-2b1c-43da-8b1f-5219554b130e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:05.069367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f184a31-2b1c-43da-8b1f-5219554b130e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:05.085023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f184a31-2b1c-43da-8b1f-5219554b130e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:05.087087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f184a31-2b1c-43da-8b1f-5219554b130e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:05.098244 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1197/2391] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:07.050388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:07.061524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be548f64-6be7-48cb-b2c8-c90d00fc7041 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:07.067182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc0bb6-2ace-49d8-992b-6c2286fd66e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:07.067849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0bb6-2ace-49d8-992b-6c2286fd66e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.068312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0bb6-2ace-49d8-992b-6c2286fd66e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.068601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0bb6-2ace-49d8-992b-6c2286fd66e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.071363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2cfc0bb6-2ace-49d8-992b-6c2286fd66e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:07.082448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc0bb6-2ace-49d8-992b-6c2286fd66e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:07.083973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc0bb6-2ace-49d8-992b-6c2286fd66e9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:07.095447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1195/2392] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:07.263879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa4df-a08c-4c39-a643-dc634d7251c5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.264350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa4df-a08c-4c39-a643-dc634d7251c5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.264446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa4df-a08c-4c39-a643-dc634d7251c5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.265858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1198/2393] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 14:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:01:07.280583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeddbd-1b20-48fd-95cc-0367ed081d97 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.281638 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeddbd-1b20-48fd-95cc-0367ed081d97 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.282088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeddbd-1b20-48fd-95cc-0367ed081d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.391177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f3aeddbd-1b20-48fd-95cc-0367ed081d97 None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:07.393267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f3aeddbd-1b20-48fd-95cc-0367ed081d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:07.406313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1196/2394] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:07.429239 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafa4df-a08c-4c39-a643-dc634d7251c5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:07.432735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efafa4df-a08c-4c39-a643-dc634d7251c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:07.461443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafa4df-a08c-4c39-a643-dc634d7251c5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:07.472054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03e386-6603-4c6e-97f3-875c2cd82acc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:07.473184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e386-6603-4c6e-97f3-875c2cd82acc service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.473651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e386-6603-4c6e-97f3-875c2cd82acc service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.475272 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e386-6603-4c6e-97f3-875c2cd82acc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.502312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03e386-6603-4c6e-97f3-875c2cd82acc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:07.505543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03e386-6603-4c6e-97f3-875c2cd82acc service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:07.533760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1199/2395] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:07.617185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaca68-b6f3-49f9-b754-d180a63e1c0a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.617763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaca68-b6f3-49f9-b754-d180a63e1c0a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.618066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaca68-b6f3-49f9-b754-d180a63e1c0a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.619670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1197/2396] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 14:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:01:07.625875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45463e-cda0-416a-805c-7a5aec0a5a02 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.626201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45463e-cda0-416a-805c-7a5aec0a5a02 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.626479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45463e-cda0-416a-805c-7a5aec0a5a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.668818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1b45463e-cda0-416a-805c-7a5aec0a5a02 None None] MFA Rules not processed for user `79d624b7de50480bb4002eb23ec2b360`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:07.670051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b45463e-cda0-416a-805c-7a5aec0a5a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:07.679641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1200/2397] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:07.693188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aaca68-b6f3-49f9-b754-d180a63e1c0a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:07.711045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aaca68-b6f3-49f9-b754-d180a63e1c0a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:07.715871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b31846-dece-4e62-a76f-c825bee38d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:07.716585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b31846-dece-4e62-a76f-c825bee38d32 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.716905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b31846-dece-4e62-a76f-c825bee38d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.717265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b31846-dece-4e62-a76f-c825bee38d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.734060 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b31846-dece-4e62-a76f-c825bee38d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:07.735602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b31846-dece-4e62-a76f-c825bee38d32 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:07.739697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:07.746527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1198/2398] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:07.885733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b37b8c-9549-4a69-863c-b77bfea97d6c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.886293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b37b8c-9549-4a69-863c-b77bfea97d6c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.886637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b37b8c-9549-4a69-863c-b77bfea97d6c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.888716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1201/2399] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 14:01:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:01:07.895026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.895315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.895962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.936662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:07.938028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:07.948283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1199/2400] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:01:07.963859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b37b8c-9549-4a69-863c-b77bfea97d6c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:07.975669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b37b8c-9549-4a69-863c-b77bfea97d6c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:07.980595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:07.981427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:07.981922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:07.982309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:07.993166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:07.994606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:08.005879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1202/2401] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:01:08.071741 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:08.073179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:08.142131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be5826-e0f5-4be7-b898-27fecef91b43 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:08.148641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:08.149359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:08.149652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:08.149937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:08.151760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:08.163897 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:08.165708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:08.177018 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1200/2402] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:11.185771 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:11.195114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67347d7e-1c84-453c-a0df-64a6b73a8cc4 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:11.198768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cb554-21b0-4dfe-83cc-c4cf0add8962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:11.199411 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cb554-21b0-4dfe-83cc-c4cf0add8962 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:11.199698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cb554-21b0-4dfe-83cc-c4cf0add8962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:11.199978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cb554-21b0-4dfe-83cc-c4cf0add8962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:11.201780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-df9cb554-21b0-4dfe-83cc-c4cf0add8962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:11.268539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cb554-21b0-4dfe-83cc-c4cf0add8962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:11.270637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cb554-21b0-4dfe-83cc-c4cf0add8962 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:11.280729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1203/2403] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:12.203292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:12.204822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:12.221247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7963029-72f9-4b74-9300-4efeae2dcbe4 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:12.227582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:12.228242 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:12.228533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:12.228818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:12.231078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:12.299537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:12.301495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:12.316393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1201/2404] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:01:17.515661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e28f3-c093-4cf0-befe-a3f1878930ac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:17.516082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e28f3-c093-4cf0-befe-a3f1878930ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:17.516249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e28f3-c093-4cf0-befe-a3f1878930ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:17.534140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f0e28f3-c093-4cf0-befe-a3f1878930ac None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:17.535396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f0e28f3-c093-4cf0-befe-a3f1878930ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:17.543531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1204/2405] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 14:01:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:17.555785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:17.567177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42284af-7939-4041-8d76-6745d231e726 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:17.572121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:17.572869 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:17.573183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:17.573467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:17.581781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:17.583530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:17.592427 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1202/2406] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:01:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:01:27.528950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:27.588541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=70 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:36.478127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e28f3-c093-4cf0-befe-a3f1878930ac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:36.488020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e28f3-c093-4cf0-befe-a3f1878930ac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:36.492084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e7a7a-6c8f-48bc-842f-4500242682ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:36.492734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7a7a-6c8f-48bc-842f-4500242682ec service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:36.493016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7a7a-6c8f-48bc-842f-4500242682ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:36.493324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7a7a-6c8f-48bc-842f-4500242682ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:36.495131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f0e7a7a-6c8f-48bc-842f-4500242682ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:36.507796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e7a7a-6c8f-48bc-842f-4500242682ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:36.509301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e7a7a-6c8f-48bc-842f-4500242682ec service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:36.517772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1205/2407] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:41.999809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:42.062962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c9b17-4fd0-4bfc-88f6-04c2ab1f576f service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:42.067372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:42.068254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.068521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.068793 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] PATH_INFO: `/v3/users/d6d5ab170e964086a00f2fdaade4a25a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.070556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d5ab170e964086a00f2fdaade4a25a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:42.072467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:42.118407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d5ab170e964086a00f2fdaade4a25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.130118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d5ab170e964086a00f2fdaade4a25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.130794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d5ab170e964086a00f2fdaade4a25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.131396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d5ab170e964086a00f2fdaade4a25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.138106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d5ab170e964086a00f2fdaade4a25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.150725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1203/2408] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:01:41 2022] DELETE /identity/v3/users/d6d5ab170e964086a00f2fdaade4a25a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:42.155160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.155656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.155762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.173716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:42.174948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:42.234433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1206/2409] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:42.244691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:42.246118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:42.305381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78bcc2-f5ba-4ee1-81c8-5e07d15fef3d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:42.308983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f41ae9-c6d0-48ff-969b-c8bb134b3457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:42.309657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f41ae9-c6d0-48ff-969b-c8bb134b3457 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.309942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f41ae9-c6d0-48ff-969b-c8bb134b3457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.310218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f41ae9-c6d0-48ff-969b-c8bb134b3457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.319054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f41ae9-c6d0-48ff-969b-c8bb134b3457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:42.320511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f41ae9-c6d0-48ff-969b-c8bb134b3457 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:42.329822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1204/2410] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:42.619398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:42.621112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:42.633908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4f09c-7c9d-4e2f-b5b9-4543932e2683 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:42.638781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:42.639769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.640305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.640674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] PATH_INFO: `/v3/projects/2e57d6db85a644e38660e2d0e3016a94` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.642112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e57d6db85a644e38660e2d0e3016a94)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:42.644125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:42.698500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e57d6db85a644e38660e2d0e3016a94', 'request_id': 'req-f7f010d4-da90-4711-a93d-b38f826a1a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.706167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e57d6db85a644e38660e2d0e3016a94', 'request_id': 'req-f7f010d4-da90-4711-a93d-b38f826a1a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.713681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e57d6db85a644e38660e2d0e3016a94', 'request_id': 'req-f7f010d4-da90-4711-a93d-b38f826a1a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.724775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] The token cache is being invalidate because project 2e57d6db85a644e38660e2d0e3016a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:01:42.725065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.726307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7f010d4-da90-4711-a93d-b38f826a1a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:42.727911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1207/2411] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:01:42 2022] DELETE /identity/v3/projects/2e57d6db85a644e38660e2d0e3016a94 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:42.733504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5ace-f970-4a42-b942-15031eedb1ac None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.733973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5ace-f970-4a42-b942-15031eedb1ac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.734414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5ace-f970-4a42-b942-15031eedb1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.752440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e8cf5ace-f970-4a42-b942-15031eedb1ac None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:42.753681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e8cf5ace-f970-4a42-b942-15031eedb1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:42.770362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:42.815096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1205/2412] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:01:42.822824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23554dc-5190-40a9-8d50-3efefcb746d8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.823116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23554dc-5190-40a9-8d50-3efefcb746d8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.823402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23554dc-5190-40a9-8d50-3efefcb746d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.843376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c23554dc-5190-40a9-8d50-3efefcb746d8 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:42.844579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c23554dc-5190-40a9-8d50-3efefcb746d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:42.853021 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1208/2413] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:42.858719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf5ace-f970-4a42-b942-15031eedb1ac None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:42.868555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf5ace-f970-4a42-b942-15031eedb1ac None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:42.873115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:42.873802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.874085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.874363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.875102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:01:42.875405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:42.877007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:42.889688 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1206/2414] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:01:42.894215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23554dc-5190-40a9-8d50-3efefcb746d8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:42.906384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23554dc-5190-40a9-8d50-3efefcb746d8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:42.911924 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:42.912589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:42.912873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:42.913230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:42.913948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:42.915884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:42.942406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1209/2415] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:01:42 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:42.950133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:42.960828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a89975-8b21-45a8-8f39-5b9bb291a4de admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:43.018860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:43.019808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:43.019808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:43.020117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:43.020962 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:43.023721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:43.044119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1207/2416] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:01:42 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:43.048324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:43.061195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dc714-ca00-4eba-86a5-f5714674d15a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:43.065643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:43.066323 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:43.066644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:43.066955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:43.067692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:43.070261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:43.081028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1210/2417] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:01:43.084818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:43.095526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c67f7-6e6e-4732-8413-7f25a5c75d84 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:43.100201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1b60f-933d-45ca-a1fe-a2e507ff664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:43.100907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b60f-933d-45ca-a1fe-a2e507ff664d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:43.101216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b60f-933d-45ca-a1fe-a2e507ff664d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:43.101497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1b60f-933d-45ca-a1fe-a2e507ff664d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:43.102228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1b60f-933d-45ca-a1fe-a2e507ff664d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:43.103813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1b60f-933d-45ca-a1fe-a2e507ff664d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:43.114178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1208/2418] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:01:43.118367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:43.128342 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcc942-4d2a-4109-92f8-5ab74bcf78a4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:43.134491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:43.135378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:43.135658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:43.135929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] PATH_INFO: `/v3/projects/98a1b861d3d84f309ff5a1e90f7499fd/users/d3425e6afcbf4d20bf4229ba6d93c8ff/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:43.154094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a1b861d3d84f309ff5a1e90f7499fd, user_id=d3425e6afcbf4d20bf4229ba6d93c8ff, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:43.157016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:43.169947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1211/2419] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:01:43 2022] PUT /identity/v3/projects/98a1b861d3d84f309ff5a1e90f7499fd/users/d3425e6afcbf4d20bf4229ba6d93c8ff/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:43.175215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287cb9-c790-444c-9131-ef553377d44f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:43.175509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287cb9-c790-444c-9131-ef553377d44f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:43.175790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287cb9-c790-444c-9131-ef553377d44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:43.194989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6b287cb9-c790-444c-9131-ef553377d44f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:43.196714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b287cb9-c790-444c-9131-ef553377d44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:43.255553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1209/2420] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:43.265931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:43.267607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:43.277525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:43.327385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e431fc6-0624-4a33-84f8-51690f8f13ec admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:43.331042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:43.331735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:43.332020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:43.332300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:43.341445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:43.342880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:43.352166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1212/2421] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:43.960251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2177d-2524-4fea-862d-385c92814041 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:43.960546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2177d-2524-4fea-862d-385c92814041 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:43.960828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2177d-2524-4fea-862d-385c92814041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:43.988441 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4ce2177d-2524-4fea-862d-385c92814041 None None] MFA Rules not processed for user `d3425e6afcbf4d20bf4229ba6d93c8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:43.989330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4ce2177d-2524-4fea-862d-385c92814041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:44.006502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:44.062832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1210/2422] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:44.073692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:44.075107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:44.136163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ccba0-2f03-4ac2-8be3-8115da8256fd service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:44.139734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:44.140531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:44.140840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:44.141114 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:44.150284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:44.151742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:44.160641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1213/2423] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:45.540384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:45.540957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:45.540957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:45.559338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] MFA Rules not processed for user `d3425e6afcbf4d20bf4229ba6d93c8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:45.560601 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:45.567958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1211/2424] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:01:45.795556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:45.796939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:45.808489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d70f13-a6f5-4517-bb15-4288c5435c5f service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:45.811951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:45.812585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:45.812863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:45.813135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:45.814835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:45.823851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:45.825292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:45.833083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1214/2425] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:48.563077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:48.564590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:48.653903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f92a4-1688-4ea9-857b-307e735e9281 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:48.659240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee61d3-a0a2-4df0-b6d2-ea1e05d41ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:48.659889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee61d3-a0a2-4df0-b6d2-ea1e05d41ac5 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:48.660197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee61d3-a0a2-4df0-b6d2-ea1e05d41ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:48.660470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee61d3-a0a2-4df0-b6d2-ea1e05d41ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:48.662295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0bee61d3-a0a2-4df0-b6d2-ea1e05d41ac5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:48.675549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee61d3-a0a2-4df0-b6d2-ea1e05d41ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:48.677180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee61d3-a0a2-4df0-b6d2-ea1e05d41ac5 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:48.688080 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1212/2426] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:49.824377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:49.825916 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:49.842570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69784c-6112-4d48-b73d-cb3f96104ac3 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:49.848039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:49.848694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:49.848984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:49.849292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:49.851596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:49.863706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:49.865367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:49.876796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1215/2427] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:01:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:55.428007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8d4b-5792-4345-802d-1c64c4915f9e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:55.428595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8d4b-5792-4345-802d-1c64c4915f9e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:55.428931 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8d4b-5792-4345-802d-1c64c4915f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:55.447619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0fcd8d4b-5792-4345-802d-1c64c4915f9e None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:55.448735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0fcd8d4b-5792-4345-802d-1c64c4915f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:55.507096 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1213/2428] 10.222.0.91 () {58 vars in 967 bytes} [Thu May 26 14:01:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:55.530761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:55.532221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:55.601376 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9ebe4-9039-4b86-b8a8-32eb9267ae4c service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:55.606334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafa7fa-449e-4f04-9d25-5bf4ca9edad5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:55.606985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafa7fa-449e-4f04-9d25-5bf4ca9edad5 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:55.607269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafa7fa-449e-4f04-9d25-5bf4ca9edad5 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:55.607547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafa7fa-449e-4f04-9d25-5bf4ca9edad5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:55.615685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafa7fa-449e-4f04-9d25-5bf4ca9edad5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:55.617333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafa7fa-449e-4f04-9d25-5bf4ca9edad5 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:55.625487 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1216/2429] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:01:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 14:02:05.500197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=81 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:05.622129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:18.863118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd8d4b-5792-4345-802d-1c64c4915f9e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.876154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd8d4b-5792-4345-802d-1c64c4915f9e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.881517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.882173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.882456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.882731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.885086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:18.950072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.951675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.961196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1214/2430] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:28.956681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=28 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:57.759308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bfe19-48eb-4822-b7f8-2f9ac43bec7f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:57.759896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bfe19-48eb-4822-b7f8-2f9ac43bec7f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:57.759896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bfe19-48eb-4822-b7f8-2f9ac43bec7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:57.778759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8f7bfe19-48eb-4822-b7f8-2f9ac43bec7f None None] MFA Rules not processed for user `8d7bcc48c8d844e5b36b1c4e2748e766`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:57.780065 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f7bfe19-48eb-4822-b7f8-2f9ac43bec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:57.787107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1217/2431] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:02:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:03:05.037474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:03:05.049508 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0fc6-4cb5-4d6f-8bed-fbf04bbb4f97 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:03:05.055642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34976bd6-cf97-4c56-8c68-f761685ec1f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:03:05.056505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976bd6-cf97-4c56-8c68-f761685ec1f4 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:03:05.056799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976bd6-cf97-4c56-8c68-f761685ec1f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:03:05.057128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976bd6-cf97-4c56-8c68-f761685ec1f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:03:05.062579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-34976bd6-cf97-4c56-8c68-f761685ec1f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:03:05.076009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34976bd6-cf97-4c56-8c68-f761685ec1f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:03:05.077675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34976bd6-cf97-4c56-8c68-f761685ec1f4 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:03:05.088087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1215/2432] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:03:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:03:07.773549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:03:09.065739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bfe19-48eb-4822-b7f8-2f9ac43bec7f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:03:09.078937 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bfe19-48eb-4822-b7f8-2f9ac43bec7f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:03:09.084305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1f9df-7ea9-40b7-a4d9-93325160d9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:03:09.084950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1f9df-7ea9-40b7-a4d9-93325160d9fa service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:03:09.085267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1f9df-7ea9-40b7-a4d9-93325160d9fa service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:03:09.085550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1f9df-7ea9-40b7-a4d9-93325160d9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:03:09.087311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-67f1f9df-7ea9-40b7-a4d9-93325160d9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:03:09.146052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1f9df-7ea9-40b7-a4d9-93325160d9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:03:09.147643 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1f9df-7ea9-40b7-a4d9-93325160d9fa service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:03:09.157705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1218/2433] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:03:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:03:15.084303 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:03:16.521583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc62a23-7480-4931-bb87-2cb17b9ad736 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:03:16.522266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc62a23-7480-4931-bb87-2cb17b9ad736 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:03:16.522266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc62a23-7480-4931-bb87-2cb17b9ad736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:03:16.542525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-afc62a23-7480-4931-bb87-2cb17b9ad736 None None] MFA Rules not processed for user `d3425e6afcbf4d20bf4229ba6d93c8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:03:16.544209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-afc62a23-7480-4931-bb87-2cb17b9ad736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:03:16.553237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1216/2434] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:03:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:03:19.152349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:03:26.535778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:03:30.264440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e7277-8297-4d83-a2af-2ebd660d42c1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:03:30.264949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e7277-8297-4d83-a2af-2ebd660d42c1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:03:30.265043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e7277-8297-4d83-a2af-2ebd660d42c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:03:30.282531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-000e7277-8297-4d83-a2af-2ebd660d42c1 None None] MFA Rules not processed for user `592975b43454407abde6c7cab2504982`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:03:30.283719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-000e7277-8297-4d83-a2af-2ebd660d42c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:03:30.290740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1219/2435] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:03:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:03:40.277082 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:03:58.228035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb859d-0d73-47ee-9af5-66741e360fe9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:03:58.228663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb859d-0d73-47ee-9af5-66741e360fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:03:58.228663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb859d-0d73-47ee-9af5-66741e360fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:03:58.247781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-70fb859d-0d73-47ee-9af5-66741e360fe9 None None] MFA Rules not processed for user `c67205b7150c460a81ff545d5ba29cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:03:58.249036 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-70fb859d-0d73-47ee-9af5-66741e360fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:03:58.304524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1217/2436] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:03:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:04:08.297983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=25 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:19.119171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a009fc-38db-4e1c-bb7d-17bb047447c2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:19.119745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a009fc-38db-4e1c-bb7d-17bb047447c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:19.119848 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a009fc-38db-4e1c-bb7d-17bb047447c2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:19.121884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1220/2437] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 14:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:04:19.130818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae87e-0ec8-45f4-bb90-f7b2ac31a5f0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:19.131687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae87e-0ec8-45f4-bb90-f7b2ac31a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:19.131963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae87e-0ec8-45f4-bb90-f7b2ac31a5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:19.158052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-151ae87e-0ec8-45f4-bb90-f7b2ac31a5f0 None None] MFA Rules not processed for user `79d624b7de50480bb4002eb23ec2b360`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:04:19.159422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-151ae87e-0ec8-45f4-bb90-f7b2ac31a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:19.166633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1218/2438] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 14:04:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:04:19.186882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a009fc-38db-4e1c-bb7d-17bb047447c2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:19.201223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a009fc-38db-4e1c-bb7d-17bb047447c2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:19.205165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c6170-3944-4d73-9529-349f86499063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:19.205822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6170-3944-4d73-9529-349f86499063 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:19.206108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6170-3944-4d73-9529-349f86499063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:19.206385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c6170-3944-4d73-9529-349f86499063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:19.215111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c6170-3944-4d73-9529-349f86499063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:19.216904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c6170-3944-4d73-9529-349f86499063 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:19.226397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1221/2439] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:04:19.375926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784a6d-564d-4d07-963b-e13002c68c8e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:19.376222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784a6d-564d-4d07-963b-e13002c68c8e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:19.376507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784a6d-564d-4d07-963b-e13002c68c8e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:19.377928 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1219/2440] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 14:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:04:19.383456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:19.383760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:19.384043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:19.402881 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:04:19.404663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:19.414240 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1222/2441] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 14:04:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:04:19.425492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d784a6d-564d-4d07-963b-e13002c68c8e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:19.436784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d784a6d-564d-4d07-963b-e13002c68c8e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:19.440650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69104-2ba0-4e40-bb18-8338a2da5919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:19.441351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69104-2ba0-4e40-bb18-8338a2da5919 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:19.441668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69104-2ba0-4e40-bb18-8338a2da5919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:19.441948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69104-2ba0-4e40-bb18-8338a2da5919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:19.452276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69104-2ba0-4e40-bb18-8338a2da5919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:19.453927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69104-2ba0-4e40-bb18-8338a2da5919 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:19.466128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1220/2442] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:04:22.050907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:22.052369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:22.157128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7743f3c-05d0-4744-bc56-176e60fe2695 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:22.165301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:22.166100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:22.166184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:22.166585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:22.168605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:22.194534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:22.201327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:22.223563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1223/2443] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:04:24.615496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0da2-6505-49e1-b3fa-d70a3f012c2d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:24.616055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0da2-6505-49e1-b3fa-d70a3f012c2d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:24.616055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0da2-6505-49e1-b3fa-d70a3f012c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:24.636866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b68f0da2-6505-49e1-b3fa-d70a3f012c2d None None] MFA Rules not processed for user `5068f06b1a8e47b88ba3294169545aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:04:24.640158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b68f0da2-6505-49e1-b3fa-d70a3f012c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:24.696355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1221/2444] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:04:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:04:27.308530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:27.313066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:27.336024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00933cd-3a94-44fd-b7bf-7151634b92c5 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:27.342657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:27.344533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:27.344533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:27.345465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:27.351126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:27.371601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:27.374483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:27.389368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1224/2445] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:04:27.392186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f0da2-6505-49e1-b3fa-d70a3f012c2d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:27.409374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f0da2-6505-49e1-b3fa-d70a3f012c2d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:27.417072 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:27.419280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:27.421101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:27.423411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] PATH_INFO: `/v3/users/d3425e6afcbf4d20bf4229ba6d93c8ff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:27.428310 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3425e6afcbf4d20bf4229ba6d93c8ff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:27.448634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:27.554808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3425e6afcbf4d20bf4229ba6d93c8ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:27.581746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3425e6afcbf4d20bf4229ba6d93c8ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:27.582970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3425e6afcbf4d20bf4229ba6d93c8ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:27.584881 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3425e6afcbf4d20bf4229ba6d93c8ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:27.601734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3425e6afcbf4d20bf4229ba6d93c8ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:27.618997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1222/2446] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:04:27 2022] DELETE /identity/v3/users/d3425e6afcbf4d20bf4229ba6d93c8ff => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:04:28.297909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:28.313361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a699f-467e-4364-8146-13999dfc9287 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:28.393546 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:28.394607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:28.394891 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:28.395170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] PATH_INFO: `/v3/projects/98a1b861d3d84f309ff5a1e90f7499fd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:28.396406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a1b861d3d84f309ff5a1e90f7499fd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:28.398364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:28.456309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a1b861d3d84f309ff5a1e90f7499fd', 'request_id': 'req-36b7123c-6ad9-405b-a0d0-c94411a3c42d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:28.466049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a1b861d3d84f309ff5a1e90f7499fd', 'request_id': 'req-36b7123c-6ad9-405b-a0d0-c94411a3c42d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:28.469473 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:28.474742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a1b861d3d84f309ff5a1e90f7499fd', 'request_id': 'req-36b7123c-6ad9-405b-a0d0-c94411a3c42d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:28.488071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] The token cache is being invalidate because project 98a1b861d3d84f309ff5a1e90f7499fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:04:28.488368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:28.489020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:28.498668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1225/2447] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:04:28 2022] DELETE /identity/v3/projects/98a1b861d3d84f309ff5a1e90f7499fd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:04:37.615327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:38.484900 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=4,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:40.382807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:40.384358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:40.461709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeefe3-f6d1-4db4-853e-09c91d569eb6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:40.539732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:40.540380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:40.540652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:40.540924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:40.542743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:40.602100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:40.603705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:40.613520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1223/2448] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:04:50.609300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:53.456419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:53.457851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:53.514894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7123c-6ad9-405b-a0d0-c94411a3c42d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:53.518907 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:53.519701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:53.520031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:53.520377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:53.522300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:53.584878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:53.586364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:53.596604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1226/2449] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:04:56.756228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:56.757706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:56.816453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:56.817932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:56.844363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3ada-0c4c-40c7-ac92-0d77caae5b4b service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:56.849113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:56.850380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:56.850697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:56.850976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] PATH_INFO: `/v3/users/8d7bcc48c8d844e5b36b1c4e2748e766` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:56.852189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7bcc48c8d844e5b36b1c4e2748e766)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:56.854199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:56.854841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e24365-0d4e-474b-8cc5-e97e0fd983e5 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:56.860554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:56.861394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:56.861639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:56.862005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:56.863696 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:56.876745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:56.878571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:56.890458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1224/2450] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:04:56.907420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7bcc48c8d844e5b36b1c4e2748e766'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:56.916614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7bcc48c8d844e5b36b1c4e2748e766'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:56.932081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7bcc48c8d844e5b36b1c4e2748e766'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:56.932778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7bcc48c8d844e5b36b1c4e2748e766'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:56.933419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7bcc48c8d844e5b36b1c4e2748e766'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:56.946092 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1227/2451] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:04:56 2022] DELETE /identity/v3/users/8d7bcc48c8d844e5b36b1c4e2748e766 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:04:57.123681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.135360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eb618-8729-41de-8750-6c7ff6a83e41 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.198764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.199501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.200294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:57.200436 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.200754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.201026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] PATH_INFO: `/v3/projects/e3d27b9eb0ef4c338c18c2a1d02db444` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.202236 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d27b9eb0ef4c338c18c2a1d02db444)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.204146 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.228144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:57.254949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d27b9eb0ef4c338c18c2a1d02db444', 'request_id': 'req-6430041b-0837-4ffe-8fc6-c3e2526201cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:57.261142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d27b9eb0ef4c338c18c2a1d02db444', 'request_id': 'req-6430041b-0837-4ffe-8fc6-c3e2526201cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:57.267549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d27b9eb0ef4c338c18c2a1d02db444', 'request_id': 'req-6430041b-0837-4ffe-8fc6-c3e2526201cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:57.280382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] The token cache is being invalidate because project e3d27b9eb0ef4c338c18c2a1d02db444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:04:57.280587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964d3f-5646-42e6-ae00-8834a6e8d514 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.280707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:57.281332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6430041b-0837-4ffe-8fc6-c3e2526201cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:04:57.282938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1225/2452] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:04:57 2022] DELETE /identity/v3/projects/e3d27b9eb0ef4c338c18c2a1d02db444 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:04:57.288408 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e0f9-f5bb-4abe-a691-5e3d820a3c89 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.288694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e0f9-f5bb-4abe-a691-5e3d820a3c89 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.288975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e0f9-f5bb-4abe-a691-5e3d820a3c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.307857 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8581e0f9-f5bb-4abe-a691-5e3d820a3c89 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:04:57.309082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8581e0f9-f5bb-4abe-a691-5e3d820a3c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:57.322773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:57.345840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.346446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.346730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.347007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.348791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:57.371307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1226/2453] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:04:57.379090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76aeb1-3efe-45c2-be47-619ddc23afba None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.379384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76aeb1-3efe-45c2-be47-619ddc23afba None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.379664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76aeb1-3efe-45c2-be47-619ddc23afba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.397275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4a76aeb1-3efe-45c2-be47-619ddc23afba None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:04:57.398471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4a76aeb1-3efe-45c2-be47-619ddc23afba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:57.406489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1227/2454] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:04:57.408620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.410350 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.410475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76aeb1-3efe-45c2-be47-619ddc23afba None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.420449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76aeb1-3efe-45c2-be47-619ddc23afba None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.422009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1228/2455] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:04:57.426052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.426746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.427032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.427311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.428340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:04:57.428445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.430085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.443719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1228/2456] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:04:57.448201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.459040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56090f17-5dcb-4756-9583-8bf0fd29bd85 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.463547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.464207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.464490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.464769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.465483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.467416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.507961 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1229/2457] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:04:57 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:04:57.509127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.519888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cde1a-43fb-452c-b7a6-0379401549aa admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.578521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.579172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.579451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.579729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.580410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.583961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.603106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1229/2458] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:04:57 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:04:57.608187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.619175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05654e1e-30bc-41a5-9a9d-18bb67de83f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.624099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.624783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.625066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.625395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.626130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.627747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.642567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1230/2459] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:04:57.644081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.655124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866a61b-1f51-42e2-8a6e-882ae4723c30 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.659964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c4797-ab68-4c48-beb8-b7a6c64ebe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.660656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4797-ab68-4c48-beb8-b7a6c64ebe4e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.660978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4797-ab68-4c48-beb8-b7a6c64ebe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.661294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4797-ab68-4c48-beb8-b7a6c64ebe4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.662041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c4797-ab68-4c48-beb8-b7a6c64ebe4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.663670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c4797-ab68-4c48-beb8-b7a6c64ebe4e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.678656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1230/2460] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:04:57.679923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.690445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d5f7d-1460-4901-abee-7331b5a3602f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.695398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.696392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.696778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.697123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] PATH_INFO: `/v3/projects/3fcd25a6c4574894a8270e3a43af782d/users/18906ec812ac49b8ba849775215c4318/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.710351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:57.713908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fcd25a6c4574894a8270e3a43af782d, user_id=18906ec812ac49b8ba849775215c4318, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.717450 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.731817 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1231/2461] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:04:57 2022] PUT /identity/v3/projects/3fcd25a6c4574894a8270e3a43af782d/users/18906ec812ac49b8ba849775215c4318/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:04:57.737792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6b99-93c6-4c07-8789-a60341611ec2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.738175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6b99-93c6-4c07-8789-a60341611ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.738529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6b99-93c6-4c07-8789-a60341611ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.757552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-94bb6b99-93c6-4c07-8789-a60341611ec2 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:04:57.759193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-94bb6b99-93c6-4c07-8789-a60341611ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:57.821810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1231/2462] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:04:57.831454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:04:57.833289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:57.894122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355c86d-6448-4502-b00d-3b3d4e8e6cb5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:04:57.897729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:04:57.898888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:57.898888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:57.898888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:57.908502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:04:57.909965 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:04:57.920693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1232/2463] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:05:02.822224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a154b9-ca23-40e8-8957-bc80d3a7271d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:02.822771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a154b9-ca23-40e8-8957-bc80d3a7271d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:02.822771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a154b9-ca23-40e8-8957-bc80d3a7271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:02.865494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b9a154b9-ca23-40e8-8957-bc80d3a7271d None None] MFA Rules not processed for user `18906ec812ac49b8ba849775215c4318`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:02.866824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b9a154b9-ca23-40e8-8957-bc80d3a7271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:02.876873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:02.941347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1232/2464] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:05:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:05:02.955873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:02.957237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:03.078459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736396f4-d65b-4ef5-b796-7fe24abf6426 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:03.084744 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7f784-07a7-413e-957d-09094cfdadde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:03.085743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7f784-07a7-413e-957d-09094cfdadde service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:03.086399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7f784-07a7-413e-957d-09094cfdadde service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:03.086847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7f784-07a7-413e-957d-09094cfdadde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:03.097126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7f784-07a7-413e-957d-09094cfdadde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:03.098810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7f784-07a7-413e-957d-09094cfdadde service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:03.111667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1233/2465] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:05:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:05:04.584864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b00c8-3357-464f-b564-595883513f68 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:04.585476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b00c8-3357-464f-b564-595883513f68 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:04.585592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b00c8-3357-464f-b564-595883513f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:04.604668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-090b00c8-3357-464f-b564-595883513f68 None None] MFA Rules not processed for user `18906ec812ac49b8ba849775215c4318`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:04.606183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-090b00c8-3357-464f-b564-595883513f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:04.617352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1233/2466] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:05:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:05:08.970994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:08.971478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:08.971594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:08.989331 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:08.990570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:09.053163 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1234/2467] 10.222.0.91 () {58 vars in 967 bytes} [Thu May 26 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:05:09.111434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b00c8-3357-464f-b564-595883513f68 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:09.112889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-090b00c8-3357-464f-b564-595883513f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:09.215490 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b00c8-3357-464f-b564-595883513f68 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:09.221722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:09.222610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:09.222898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:09.223179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:09.233251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:09.234913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:09.244212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1234/2468] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:05:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:05:09.607971 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:09.610661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:09.623044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eaba6a-1cfa-4635-b956-cdc9be87c73c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:09.626864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:09.627527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:09.627814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:09.628093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:09.629958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:09.644340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:09.645857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:09.657421 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1235/2469] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:05:16.467922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:16.469295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:16.492701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fca2f1-eec7-42a6-8975-a51049729bdf service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:16.497438 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:16.498319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:16.498602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:16.498873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] PATH_INFO: `/v3/users/0e7ef003bc5642feadfa1f9f631ef903` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:16.500044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7ef003bc5642feadfa1f9f631ef903)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:16.502014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:16.556030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7ef003bc5642feadfa1f9f631ef903'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.569258 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7ef003bc5642feadfa1f9f631ef903'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.569970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7ef003bc5642feadfa1f9f631ef903'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.570582 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7ef003bc5642feadfa1f9f631ef903'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.578942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7ef003bc5642feadfa1f9f631ef903'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.594319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1235/2470] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:05:16 2022] DELETE /identity/v3/users/0e7ef003bc5642feadfa1f9f631ef903 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:16.767637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:16.778136 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526331-b054-4737-9fcb-2f8c014550c2 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:16.786676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:16.837627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:16.838495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:16.838785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:16.839066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] PATH_INFO: `/v3/projects/08ed6980f1214f6789f94f34a4c68e6e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:16.840266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ed6980f1214f6789f94f34a4c68e6e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:16.842287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:16.888311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ed6980f1214f6789f94f34a4c68e6e', 'request_id': 'req-7a654d9f-464d-4278-a28d-5c52015038df'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.896534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ed6980f1214f6789f94f34a4c68e6e', 'request_id': 'req-7a654d9f-464d-4278-a28d-5c52015038df'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.902748 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ed6980f1214f6789f94f34a4c68e6e', 'request_id': 'req-7a654d9f-464d-4278-a28d-5c52015038df'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.914126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] The token cache is being invalidate because project 08ed6980f1214f6789f94f34a4c68e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:05:16.914414 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.915067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:16.916650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1236/2471] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:05:16 2022] DELETE /identity/v3/projects/08ed6980f1214f6789f94f34a4c68e6e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:16.921238 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:16.922656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:16.933041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:16.939932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8a216-f459-4dd3-abbf-f241017d785e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:17.009051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:17.009966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:17.010305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:17.010540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] PATH_INFO: `/v3/users/e530500987d3441ba10c02c94c92098c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:17.011726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e530500987d3441ba10c02c94c92098c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:17.014210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:17.065644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e530500987d3441ba10c02c94c92098c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.077997 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e530500987d3441ba10c02c94c92098c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.078694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e530500987d3441ba10c02c94c92098c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.079294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e530500987d3441ba10c02c94c92098c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.086889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596048a5-ec3d-4104-9175-dec0a3acad9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e530500987d3441ba10c02c94c92098c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.101493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1236/2472] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:05:16 2022] DELETE /identity/v3/users/e530500987d3441ba10c02c94c92098c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:17.316648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:17.318080 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:17.330437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a654d9f-464d-4278-a28d-5c52015038df admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:17.405177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:17.406059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:17.406343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:17.406621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] PATH_INFO: `/v3/projects/60a6f4f6d3854690befed268a5911a3b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:17.407800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a6f4f6d3854690befed268a5911a3b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:17.410040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:17.454503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:17.459408 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a6f4f6d3854690befed268a5911a3b', 'request_id': 'req-fcb39054-01a0-42d2-bb13-b3b42b359960'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.467818 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a6f4f6d3854690befed268a5911a3b', 'request_id': 'req-fcb39054-01a0-42d2-bb13-b3b42b359960'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.474116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a6f4f6d3854690befed268a5911a3b', 'request_id': 'req-fcb39054-01a0-42d2-bb13-b3b42b359960'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.487033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] The token cache is being invalidate because project 60a6f4f6d3854690befed268a5911a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:05:17.487739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.488492 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:17.490397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1237/2473] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:05:17 2022] DELETE /identity/v3/projects/60a6f4f6d3854690befed268a5911a3b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:27.097527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:27.481363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:42.974961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e12f77-5147-4ef2-b589-b3edb6a03a8e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:42.975502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e12f77-5147-4ef2-b589-b3edb6a03a8e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:42.975621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e12f77-5147-4ef2-b589-b3edb6a03a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:42.992690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e1e12f77-5147-4ef2-b589-b3edb6a03a8e None None] MFA Rules not processed for user `592975b43454407abde6c7cab2504982`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:42.993994 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e1e12f77-5147-4ef2-b589-b3edb6a03a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:43.048196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1237/2474] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:05:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:05:53.041729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=25 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:04.149513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:04.150947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:04.209790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb39054-01a0-42d2-bb13-b3b42b359960 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:04.213513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:04.214164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:04.214453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:04.214732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:04.217089 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:04.286928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:04.288356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:04.299222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1238/2475] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:05.474055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e12f77-5147-4ef2-b589-b3edb6a03a8e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:05.484106 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e12f77-5147-4ef2-b589-b3edb6a03a8e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:05.487625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cca31a-d140-47a4-af2a-44f91ac60083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:05.488263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca31a-d140-47a4-af2a-44f91ac60083 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:05.488546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca31a-d140-47a4-af2a-44f91ac60083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:05.488822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca31a-d140-47a4-af2a-44f91ac60083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:05.490631 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-38cca31a-d140-47a4-af2a-44f91ac60083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:05.504400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cca31a-d140-47a4-af2a-44f91ac60083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:05.505866 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cca31a-d140-47a4-af2a-44f91ac60083 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:05.517254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1238/2476] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:12.311062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:12.312440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:12.367994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b03cd-8eb2-498f-b256-c3978bc4141d service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:12.371804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:12.372458 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:12.372774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:12.373053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:12.374874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:12.431195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:12.455783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:12.457262 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:12.476026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1239/2477] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:12.584010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86247fc8-60e4-42aa-ae05-054683e977ee None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:12.584554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86247fc8-60e4-42aa-ae05-054683e977ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:12.584675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86247fc8-60e4-42aa-ae05-054683e977ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:12.603074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-86247fc8-60e4-42aa-ae05-054683e977ee None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:06:12.604280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-86247fc8-60e4-42aa-ae05-054683e977ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:12.661734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1239/2478] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 14:06:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:06:12.708097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:12.709551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:12.779424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b644e-36a2-47b3-bed8-dfa6254fd4c8 service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:12.784679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea87ee7-d5cd-4eea-984e-1b6f76f258dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:12.785352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea87ee7-d5cd-4eea-984e-1b6f76f258dd service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:12.786303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea87ee7-d5cd-4eea-984e-1b6f76f258dd service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:12.786303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea87ee7-d5cd-4eea-984e-1b6f76f258dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:12.794073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea87ee7-d5cd-4eea-984e-1b6f76f258dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:12.795696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea87ee7-d5cd-4eea-984e-1b6f76f258dd service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:12.805456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1240/2479] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:06:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:06:22.655131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=31 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:22.800733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=47 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:28.786672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86247fc8-60e4-42aa-ae05-054683e977ee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:28.796533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86247fc8-60e4-42aa-ae05-054683e977ee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:28.800061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4dc08-6311-42f9-b4a1-00754ca77a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:28.800731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4dc08-6311-42f9-b4a1-00754ca77a48 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:28.801013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4dc08-6311-42f9-b4a1-00754ca77a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:28.801319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4dc08-6311-42f9-b4a1-00754ca77a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:28.803084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b1d4dc08-6311-42f9-b4a1-00754ca77a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:28.873907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4dc08-6311-42f9-b4a1-00754ca77a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:28.875349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4dc08-6311-42f9-b4a1-00754ca77a48 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:28.884802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1240/2480] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:30.455926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51e60-c9ab-49c4-8151-2188898b8c06 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:30.456588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51e60-c9ab-49c4-8151-2188898b8c06 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:30.457030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51e60-c9ab-49c4-8151-2188898b8c06 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:30.459385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1241/2481] 10.222.0.91 () {54 vars in 921 bytes} [Thu May 26 14:06:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:06:30.464684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:30.465899 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:30.466011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:30.486124 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] MFA Rules not processed for user `f579f2db041f455786a02dcf621619ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:06:30.487477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:30.497772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1241/2482] 10.222.0.91 () {58 vars in 1018 bytes} [Thu May 26 14:06:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:06:30.510133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51e60-c9ab-49c4-8151-2188898b8c06 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:30.523630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51e60-c9ab-49c4-8151-2188898b8c06 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:30.529107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f88535-fbd8-4706-9b6b-bfc3cf77fe03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c209755dfc4b4089176a056cf2f683', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:30.530203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f88535-fbd8-4706-9b6b-bfc3cf77fe03 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:30.530706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f88535-fbd8-4706-9b6b-bfc3cf77fe03 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:30.531415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f88535-fbd8-4706-9b6b-bfc3cf77fe03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:30.549292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f88535-fbd8-4706-9b6b-bfc3cf77fe03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:30.550755 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f88535-fbd8-4706-9b6b-bfc3cf77fe03 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:30.562762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1242/2483] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:30.795039 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:30.796543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:30.878664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcfaf3-cf46-4a57-a3ea-0f1254862ceb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:30.883850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:30.884511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:30.884800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:30.885338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:30.887900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:30.904028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:30.905904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:30.917463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1242/2484] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:33.129006 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:33.129674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:33.129674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:33.152113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:06:33.154035 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:33.164333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1243/2485] 10.222.0.91 () {58 vars in 1009 bytes} [Thu May 26 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:06:33.178909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:33.192463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68328016-dc68-4fb7-a24d-96baa6356b07 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:33.205249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:33.206641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:33.207231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:33.207599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:33.218808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:33.220429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:33.229511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1243/2486] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:06:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:06:36.858983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:36.860677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:36.899187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e8c84-fe9d-4bbf-a84a-9104bc48350f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:36.905682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:36.906572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:36.906880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:36.907149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] PATH_INFO: `/v3/users/ecacca157e3a43b79659f78e4beb0c5e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:36.908379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecacca157e3a43b79659f78e4beb0c5e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:36.910340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:36.988539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecacca157e3a43b79659f78e4beb0c5e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.000333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecacca157e3a43b79659f78e4beb0c5e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.025672 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecacca157e3a43b79659f78e4beb0c5e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.026436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecacca157e3a43b79659f78e4beb0c5e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.027086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecacca157e3a43b79659f78e4beb0c5e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.064420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1244/2487] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:06:36 2022] DELETE /identity/v3/users/ecacca157e3a43b79659f78e4beb0c5e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:06:37.339800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:37.354378 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d99bcf-f6e5-4a5b-b12a-eb84ba480996 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:37.425721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:37.426589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:37.426870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:37.427207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] PATH_INFO: `/v3/projects/4d13c9a0ad1146158525c3398ff4de2a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:37.428416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d13c9a0ad1146158525c3398ff4de2a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:37.430515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:37.437722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:37.482765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d13c9a0ad1146158525c3398ff4de2a', 'request_id': 'req-ac0f6281-0333-412f-b262-15a77d2f079b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.491473 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d13c9a0ad1146158525c3398ff4de2a', 'request_id': 'req-ac0f6281-0333-412f-b262-15a77d2f079b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.499121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d13c9a0ad1146158525c3398ff4de2a', 'request_id': 'req-ac0f6281-0333-412f-b262-15a77d2f079b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.513410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] The token cache is being invalidate because project 4d13c9a0ad1146158525c3398ff4de2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:06:37.513701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.514570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.516129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1244/2488] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:06:37 2022] DELETE /identity/v3/projects/4d13c9a0ad1146158525c3398ff4de2a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:06:37.520303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:37.521725 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:37.535956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd30730-bdd9-433a-9f8b-6064d97b04ba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:37.603851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:37.604641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:37.604922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:37.605230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] PATH_INFO: `/v3/users/e757da0c768147adb41da77866ed35c9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:37.606932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e757da0c768147adb41da77866ed35c9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:37.608839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:37.660740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e757da0c768147adb41da77866ed35c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.671131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e757da0c768147adb41da77866ed35c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.672524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:37.689727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e757da0c768147adb41da77866ed35c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.690462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e757da0c768147adb41da77866ed35c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.691064 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e757da0c768147adb41da77866ed35c9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:37.703764 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1245/2489] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:06:37 2022] DELETE /identity/v3/users/e757da0c768147adb41da77866ed35c9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:06:37.896757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:37.898324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:37.913459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f6281-0333-412f-b262-15a77d2f079b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:37.988183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:37.989664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:37.990213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:37.990750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] PATH_INFO: `/v3/projects/a5028cfaabd84d63a6ff2cb33b0133e4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:37.992395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5028cfaabd84d63a6ff2cb33b0133e4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:37.994705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:38.061985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5028cfaabd84d63a6ff2cb33b0133e4', 'request_id': 'req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:38.069130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5028cfaabd84d63a6ff2cb33b0133e4', 'request_id': 'req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:38.075122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5028cfaabd84d63a6ff2cb33b0133e4', 'request_id': 'req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:38.088646 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] The token cache is being invalidate because project a5028cfaabd84d63a6ff2cb33b0133e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:06:38.088955 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:38.089753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:06:38.092131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1245/2490] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:06:37 2022] DELETE /identity/v3/projects/a5028cfaabd84d63a6ff2cb33b0133e4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:06:47.700009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=11,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:48.085229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=73,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:06:53.740973 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:53.742465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:53.833001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd3c7-1d2b-4637-a505-d208ba007e11 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:53.897853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5502d-c65f-4e98-ba42-2eeb6184dc2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:53.898515 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5502d-c65f-4e98-ba42-2eeb6184dc2f service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:53.898796 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5502d-c65f-4e98-ba42-2eeb6184dc2f service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:53.899072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5502d-c65f-4e98-ba42-2eeb6184dc2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:53.900873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-44c5502d-c65f-4e98-ba42-2eeb6184dc2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:53.913772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5502d-c65f-4e98-ba42-2eeb6184dc2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:53.915377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5502d-c65f-4e98-ba42-2eeb6184dc2f service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:53.926801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1246/2491] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:54.158279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:54.159670 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:54.219761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e1297-127e-497d-9c5b-fd1d9dbd3f43 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:54.223531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:54.224187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:54.224472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:54.224747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:54.226593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:54.237295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:54.239649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:54.249844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1246/2492] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:06:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:06:56.415188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:56.415694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:56.415798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:56.434512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] MFA Rules not processed for user `3bf30288daec48dd8e04b96492e87341`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:06:56.435710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:56.497879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1247/2493] 10.222.0.91 () {58 vars in 967 bytes} [Thu May 26 14:06:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:06:56.510719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:06:56.512119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:06:56.583637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387ec2c-c21f-452b-ab53-8dbe2cc35026 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:06:56.589218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3e737-aeb4-4ce4-b2d3-1755fb186132 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cbe226d7639456ebb94de533de30d28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:06:56.589887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3e737-aeb4-4ce4-b2d3-1755fb186132 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:06:56.590231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3e737-aeb4-4ce4-b2d3-1755fb186132 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:06:56.590579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3e737-aeb4-4ce4-b2d3-1755fb186132 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:06:56.599337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3e737-aeb4-4ce4-b2d3-1755fb186132 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:06:56.601003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3e737-aeb4-4ce4-b2d3-1755fb186132 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:06:56.609691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1247/2494] 10.222.0.91 () {60 vars in 1384 bytes} [Thu May 26 14:06:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:07:01.946857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:01.948268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:01.979568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:02.004751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150de970-3656-47f0-9fc4-dc8c985870d7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:02.008276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:02.008913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:02.009221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:02.009493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:02.011208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:02.073386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:02.074823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:02.084281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1248/2495] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:07:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:07:06.605974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=58 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:12.080072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=35 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:13.092218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:13.092773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:13.092773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:13.110951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] MFA Rules not processed for user `5068f06b1a8e47b88ba3294169545aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:07:13.112192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:13.119073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1248/2496] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:07:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:07:13.429353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:13.430809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:13.442357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdf839-18a3-47ef-87dd-8f15401f34c0 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:13.445851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:13.446488 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:13.446770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:13.447046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:13.448762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:13.457994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:13.459410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:13.467699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1249/2497] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:07:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:07:16.414887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:16.417059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:16.436788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa40181-4188-4be1-9a08-72e7519977d5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:16.443264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:16.443945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:16.444233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:16.444515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:16.446690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:16.461607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:16.463269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:16.475887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1249/2498] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:07:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:07:23.463703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:26.470345 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=9 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:31.643808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:31.653246 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c59570-d21f-40c7-85cb-54bc39d87772 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:31.656853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:31.657527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:31.657811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:31.658087 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:31.659874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:31.724915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:31.726381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:31.737142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1250/2499] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:07:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:07:41.731652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=30 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:52.978810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:52.988029 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf42f6-8682-4ccc-9596-2fa81a321134 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:52.991634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:52.992271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:52.992556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:52.992832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:52.994632 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:53.059051 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:53.061041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:53.070850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1250/2500] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:07:56.256957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:56.258517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:56.346890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de4798-bdd1-4c3d-96fe-7baa095e2fbb service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:56.351445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:56.352316 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:56.352598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:56.352872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] PATH_INFO: `/v3/users/592975b43454407abde6c7cab2504982` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:56.365814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=592975b43454407abde6c7cab2504982)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:56.367760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:56.415718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592975b43454407abde6c7cab2504982'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.422798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592975b43454407abde6c7cab2504982'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.434354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592975b43454407abde6c7cab2504982'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.434995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592975b43454407abde6c7cab2504982'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.435663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20a2b74f-f63c-4373-b75e-162bd8b05cf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592975b43454407abde6c7cab2504982'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.447518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1251/2501] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:07:56 2022] DELETE /identity/v3/users/592975b43454407abde6c7cab2504982 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:07:56.614161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:56.673487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3dbc7-eccd-45b4-9a7f-d0ee60e120e2 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:56.728657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:56.729559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:56.729845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:56.730118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] PATH_INFO: `/v3/projects/af9d168c40ea4df99b495eb06245d066` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:56.740418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9d168c40ea4df99b495eb06245d066)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:56.742376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:56.787338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:56.788584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9d168c40ea4df99b495eb06245d066', 'request_id': 'req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.796108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9d168c40ea4df99b495eb06245d066', 'request_id': 'req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.801497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9d168c40ea4df99b495eb06245d066', 'request_id': 'req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.813911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] The token cache is being invalidate because project af9d168c40ea4df99b495eb06245d066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:07:56.814194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.814804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a655070d-1357-4b0b-8c1f-04f4b7a6c9ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:07:56.816316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1251/2502] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:07:56 2022] DELETE /identity/v3/projects/af9d168c40ea4df99b495eb06245d066 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:07:56.821362 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6b6a8-b6da-401f-937d-6ec0a91f5285 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:56.821660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6b6a8-b6da-401f-937d-6ec0a91f5285 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:56.821941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6b6a8-b6da-401f-937d-6ec0a91f5285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:56.867453 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0bb6b6a8-b6da-401f-937d-6ec0a91f5285 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:07:56.868582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0bb6b6a8-b6da-401f-937d-6ec0a91f5285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:56.901279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:56.928363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1252/2503] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:07:56.936597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4184a-93a3-4662-b937-f16448eb602f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:56.936937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4184a-93a3-4662-b937-f16448eb602f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:56.937251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4184a-93a3-4662-b937-f16448eb602f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:56.955727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e1c4184a-93a3-4662-b937-f16448eb602f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:07:56.956924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e1c4184a-93a3-4662-b937-f16448eb602f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:56.964732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1252/2504] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:07:56.968867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6b6a8-b6da-401f-937d-6ec0a91f5285 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:56.978435 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6b6a8-b6da-401f-937d-6ec0a91f5285 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:56.982902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:56.983544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:56.983825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:56.984097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:56.984813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:07:56.985117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:56.986724 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:56.998351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1253/2505] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:07:57.002731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4184a-93a3-4662-b937-f16448eb602f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:57.011796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4184a-93a3-4662-b937-f16448eb602f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:57.017471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:57.018123 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:57.018409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:57.018684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:57.019361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:57.021296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:57.048165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1253/2506] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:07:57 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:07:57.052224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:57.062627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0cd3b-3c8b-4960-90d2-b78c9c639e20 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:57.122616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:57.123214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:57.123503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:57.123778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:57.124479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:57.126562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:57.153013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1254/2507] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:07:57 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:07:57.157653 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:57.168513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8b561-1874-41de-a456-f046870341cd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:57.173816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:57.174523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:57.174970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:57.175255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:57.175995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:57.177673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:57.189429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1254/2508] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:07:57.193565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:57.204660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51646aea-b290-4970-b06a-ea6df3e4eea0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:57.209678 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eccd6-f9cc-4006-9931-a4e2f559e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:57.210391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eccd6-f9cc-4006-9931-a4e2f559e199 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:57.210759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eccd6-f9cc-4006-9931-a4e2f559e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:57.211013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eccd6-f9cc-4006-9931-a4e2f559e199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:57.212352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eccd6-f9cc-4006-9931-a4e2f559e199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:57.214223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eccd6-f9cc-4006-9931-a4e2f559e199 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:57.227831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1255/2509] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:07:57.232706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:57.242794 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cad2b2-5667-4fbc-a7cf-1a39304c02f7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:57.248131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:57.249005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:57.249330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:57.249662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] PATH_INFO: `/v3/projects/478f3f9e176544e092ed2debbe2fe0fb/users/c412cb9123c44b248e75df55ae4fb694/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:57.265483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478f3f9e176544e092ed2debbe2fe0fb, user_id=c412cb9123c44b248e75df55ae4fb694, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:57.268156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:57.280816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1255/2510] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:07:57 2022] PUT /identity/v3/projects/478f3f9e176544e092ed2debbe2fe0fb/users/c412cb9123c44b248e75df55ae4fb694/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:07:57.290790 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5ee14-b1bf-4e51-88f7-b4360adaca40 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:57.291282 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5ee14-b1bf-4e51-88f7-b4360adaca40 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:57.291380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5ee14-b1bf-4e51-88f7-b4360adaca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:57.318391 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d6c5ee14-b1bf-4e51-88f7-b4360adaca40 None None] MFA Rules not processed for user `c412cb9123c44b248e75df55ae4fb694`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:07:57.319587 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d6c5ee14-b1bf-4e51-88f7-b4360adaca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:57.388210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1256/2511] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:07:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:07:57.400974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:57.402743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:57.492079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420368a8-6f77-44ae-bc0b-c0fda00641c6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:57.497657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3ee38-827d-4ae1-bd02-5266af280095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:57.498314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3ee38-827d-4ae1-bd02-5266af280095 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:57.498598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3ee38-827d-4ae1-bd02-5266af280095 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:57.498873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3ee38-827d-4ae1-bd02-5266af280095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:57.506724 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3ee38-827d-4ae1-bd02-5266af280095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:57.508312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3ee38-827d-4ae1-bd02-5266af280095 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:57.517284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1256/2512] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:07:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:06.906481 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5ee14-b1bf-4e51-88f7-b4360adaca40 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:06.940810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:06.973125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5ee14-b1bf-4e51-88f7-b4360adaca40 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:06.977691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:06.978570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:06.978850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:06.979131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] PATH_INFO: `/v3/users/c412cb9123c44b248e75df55ae4fb694` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:06.980324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c412cb9123c44b248e75df55ae4fb694)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:06.982308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:07.032423 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c412cb9123c44b248e75df55ae4fb694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.040225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c412cb9123c44b248e75df55ae4fb694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.053512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c412cb9123c44b248e75df55ae4fb694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.054465 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c412cb9123c44b248e75df55ae4fb694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.055269 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c412cb9123c44b248e75df55ae4fb694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.068875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1257/2513] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:06 2022] DELETE /identity/v3/users/c412cb9123c44b248e75df55ae4fb694 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:07.073808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.074377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.074377 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.098952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:07.100175 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:07.147737 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:07.171947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1257/2514] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:07.182759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:07.184667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:07.271566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784c5d-5161-4463-a2e8-83d9642dc066 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:07.275403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9468e-30df-41bb-bcc3-a8b98249ec9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:07.276058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9468e-30df-41bb-bcc3-a8b98249ec9a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.276351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9468e-30df-41bb-bcc3-a8b98249ec9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.276635 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9468e-30df-41bb-bcc3-a8b98249ec9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.286214 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9468e-30df-41bb-bcc3-a8b98249ec9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:07.287624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9468e-30df-41bb-bcc3-a8b98249ec9a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:07.297333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1258/2515] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:07.584709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:07.586216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:07.599710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d3d9e-76eb-435e-aa43-804689a6f01c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:07.606197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:07.607086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.607865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.608143 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] PATH_INFO: `/v3/projects/478f3f9e176544e092ed2debbe2fe0fb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.610346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=478f3f9e176544e092ed2debbe2fe0fb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:07.611505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:07.673733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478f3f9e176544e092ed2debbe2fe0fb', 'request_id': 'req-69ed6d0b-e597-410e-8058-fb1ee4a799ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.680619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478f3f9e176544e092ed2debbe2fe0fb', 'request_id': 'req-69ed6d0b-e597-410e-8058-fb1ee4a799ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.686406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478f3f9e176544e092ed2debbe2fe0fb', 'request_id': 'req-69ed6d0b-e597-410e-8058-fb1ee4a799ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.702495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] The token cache is being invalidate because project 478f3f9e176544e092ed2debbe2fe0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:07.702790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.703450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69ed6d0b-e597-410e-8058-fb1ee4a799ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:07.705033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1258/2516] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:07 2022] DELETE /identity/v3/projects/478f3f9e176544e092ed2debbe2fe0fb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:07.711359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77cb9-49f2-46f4-91d0-36a78a5d9471 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.711653 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77cb9-49f2-46f4-91d0-36a78a5d9471 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.711955 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77cb9-49f2-46f4-91d0-36a78a5d9471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.734859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9af77cb9-49f2-46f4-91d0-36a78a5d9471 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:07.736100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9af77cb9-49f2-46f4-91d0-36a78a5d9471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:07.767593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:07.803187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1259/2517] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:07.812669 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f30bb-44aa-4f40-bf54-73b68c15ff9b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.812961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f30bb-44aa-4f40-bf54-73b68c15ff9b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.813276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f30bb-44aa-4f40-bf54-73b68c15ff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.831982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6c9f30bb-44aa-4f40-bf54-73b68c15ff9b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:07.833226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6c9f30bb-44aa-4f40-bf54-73b68c15ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:07.842195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1259/2518] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:07.846503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77cb9-49f2-46f4-91d0-36a78a5d9471 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:07.857056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77cb9-49f2-46f4-91d0-36a78a5d9471 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:07.862422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:07.863251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.863535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.863805 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.864564 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:07.864867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:07.866503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:07.878541 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1260/2519] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:07.882941 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f30bb-44aa-4f40-bf54-73b68c15ff9b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:07.892393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f30bb-44aa-4f40-bf54-73b68c15ff9b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:07.897041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:07.897728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.898013 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.898290 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.898979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:07.900886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:07.927141 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1260/2520] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:07 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:07.931249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:07.941354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874517ae-e690-469d-8d67-756fbe4bad7f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:07.996795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:07.997499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:07.997785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:07.998062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:07.998749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.000679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.030264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1261/2521] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:07 2022] POST /identity/v3/users => generated 278 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:08.034486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.044683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a03c0-9a3a-493a-a8b1-b7d72fe1ca1f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.050145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.050800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.051086 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.051366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.052092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.053729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.064100 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1261/2522] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:08.067945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.078340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0d168-1a0d-4c60-961e-794fdcec9f53 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.083003 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb6d3a-bd57-4fb4-bc3a-506ed89f5cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.083662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6d3a-bd57-4fb4-bc3a-506ed89f5cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.083950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6d3a-bd57-4fb4-bc3a-506ed89f5cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.084227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6d3a-bd57-4fb4-bc3a-506ed89f5cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.084953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb6d3a-bd57-4fb4-bc3a-506ed89f5cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.086597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb6d3a-bd57-4fb4-bc3a-506ed89f5cbc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.097067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1262/2523] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:08.101068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.112390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b12e-c31f-4c13-9384-ac9ac631979e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.117079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.117996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.118284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.118561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] PATH_INFO: `/v3/projects/568f8b139a7f4102a31db8bee764ec2d/users/0aad2e89e5974218aa0f80c2c36e9d8f/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.133302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568f8b139a7f4102a31db8bee764ec2d, user_id=0aad2e89e5974218aa0f80c2c36e9d8f, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.135971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.149566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1262/2524] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:08 2022] PUT /identity/v3/projects/568f8b139a7f4102a31db8bee764ec2d/users/0aad2e89e5974218aa0f80c2c36e9d8f/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:08.159997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ab3b-0e7f-4843-91de-66ff9bd78487 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.160295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ab3b-0e7f-4843-91de-66ff9bd78487 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.160576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ab3b-0e7f-4843-91de-66ff9bd78487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.191868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5247ab3b-0e7f-4843-91de-66ff9bd78487 None None] MFA Rules not processed for user `0aad2e89e5974218aa0f80c2c36e9d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:08.193090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5247ab3b-0e7f-4843-91de-66ff9bd78487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:08.258309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1263/2525] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:08.263157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.324652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876e96a-98f6-44bc-8bcc-1033bc04e4e6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.330112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.330770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.331054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.331331 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.332015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.333963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.338009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:08.361341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1263/2526] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:08 2022] POST /identity/v3/projects => generated 355 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:08.365367 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ab3b-0e7f-4843-91de-66ff9bd78487 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.378958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ab3b-0e7f-4843-91de-66ff9bd78487 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.408505 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:08.436681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.437388 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.437676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.437959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.438674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.440582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.459654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1264/2527] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:08 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:08.463693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.475056 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a210-e82d-4bf1-9d8f-284ce059f93a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.479691 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.480357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.480639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.480912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.481699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.483822 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.494811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1264/2528] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:08.500061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.510156 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd84939-0cd3-4cee-81b6-d558e3ab78ec admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.515131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.516152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.517048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.517048 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] PATH_INFO: `/v3/projects/5410c932dc2a4781a8e1bff061370e97/users/6efcb2c1b01b439497551d50b891191c/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.532456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5410c932dc2a4781a8e1bff061370e97, user_id=6efcb2c1b01b439497551d50b891191c, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.535153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.548786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1265/2529] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:08 2022] PUT /identity/v3/projects/5410c932dc2a4781a8e1bff061370e97/users/6efcb2c1b01b439497551d50b891191c/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:08.553225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.563988 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65205798-900b-4a3f-960a-102251e74252 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.625528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630d80-34fc-4d26-bf84-b07b49a1a6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.626202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630d80-34fc-4d26-bf84-b07b49a1a6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.626492 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630d80-34fc-4d26-bf84-b07b49a1a6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.626770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630d80-34fc-4d26-bf84-b07b49a1a6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.627500 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40630d80-34fc-4d26-bf84-b07b49a1a6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.629128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40630d80-34fc-4d26-bf84-b07b49a1a6b7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.640050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1265/2530] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:08.644333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.655292 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e590a69-ae3a-4b76-8438-b7a88530ebb1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.661069 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.661981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.662306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.662585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] PATH_INFO: `/v3/projects/5410c932dc2a4781a8e1bff061370e97/users/6efcb2c1b01b439497551d50b891191c/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.664600 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5410c932dc2a4781a8e1bff061370e97, user_id=6efcb2c1b01b439497551d50b891191c, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.667290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.678697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1266/2531] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:08 2022] PUT /identity/v3/projects/5410c932dc2a4781a8e1bff061370e97/users/6efcb2c1b01b439497551d50b891191c/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:08.689121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2090-8592-4de0-98c7-701466ea1b3a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.689453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2090-8592-4de0-98c7-701466ea1b3a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.689732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2090-8592-4de0-98c7-701466ea1b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.718298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-06bf2090-8592-4de0-98c7-701466ea1b3a None None] MFA Rules not processed for user `6efcb2c1b01b439497551d50b891191c`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:08.720245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-06bf2090-8592-4de0-98c7-701466ea1b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:08.799878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1266/2532] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:08.811501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.813359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:08.870267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710821c-845f-4a2f-a4ef-c4e4e42b3dab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.873947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.874596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.874877 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.875157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:08.884503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:08.885952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:08.896077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1267/2533] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:08 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:08.970744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2090-8592-4de0-98c7-701466ea1b3a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:08.980629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2090-8592-4de0-98c7-701466ea1b3a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:08.984861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:08.985559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:08.985847 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:08.986129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:09.044153 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:09.046726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:09.049076 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:09.059077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1267/2534] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:08 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:18.891577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:19.055203 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:21.987958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:21.989393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:22.045716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5338-6e64-48d8-9f56-15641aff44c9 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:22.049544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f085e4-9a6c-43f1-8060-393570c4d8ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:22.050224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f085e4-9a6c-43f1-8060-393570c4d8ad service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:22.050512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f085e4-9a6c-43f1-8060-393570c4d8ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:22.050788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f085e4-9a6c-43f1-8060-393570c4d8ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:22.052585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90f085e4-9a6c-43f1-8060-393570c4d8ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:22.117986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f085e4-9a6c-43f1-8060-393570c4d8ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:22.119428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f085e4-9a6c-43f1-8060-393570c4d8ad service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:22.128947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1268/2535] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:23.912687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:23.923823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a593e9-ce96-43f1-b7a0-b0e786d12d92 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:23.928317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:23.929262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:23.929549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:23.929874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] PATH_INFO: `/v3/users/0aad2e89e5974218aa0f80c2c36e9d8f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:23.931011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aad2e89e5974218aa0f80c2c36e9d8f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:23.933185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:23.987965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aad2e89e5974218aa0f80c2c36e9d8f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.003771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aad2e89e5974218aa0f80c2c36e9d8f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.004406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aad2e89e5974218aa0f80c2c36e9d8f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.005021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aad2e89e5974218aa0f80c2c36e9d8f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.015542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aad2e89e5974218aa0f80c2c36e9d8f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.037781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1268/2536] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:23 2022] DELETE /identity/v3/users/0aad2e89e5974218aa0f80c2c36e9d8f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:24.042677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:24.043520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:24.044106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:24.067722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:24.069212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:24.143131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1269/2537] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:24.153443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:24.155101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:24.225444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88334b-3186-4303-9eba-15884609feba admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:24.229113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:24.229790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:24.230073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:24.230352 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:24.241538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:24.243165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:24.255559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1269/2538] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:24.547800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:24.549290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:24.569917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba20320-92bb-44f4-a6f9-7cc8d1b2855d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:24.575913 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:24.576908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:24.577220 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:24.577502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] PATH_INFO: `/v3/projects/568f8b139a7f4102a31db8bee764ec2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:24.578706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568f8b139a7f4102a31db8bee764ec2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:24.580978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:24.643650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:24.649257 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568f8b139a7f4102a31db8bee764ec2d', 'request_id': 'req-8df83608-9d42-48ea-8042-a53e92451842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.663283 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568f8b139a7f4102a31db8bee764ec2d', 'request_id': 'req-8df83608-9d42-48ea-8042-a53e92451842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.675112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568f8b139a7f4102a31db8bee764ec2d', 'request_id': 'req-8df83608-9d42-48ea-8042-a53e92451842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.687752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] The token cache is being invalidate because project 568f8b139a7f4102a31db8bee764ec2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:24.688074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.688709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.690531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1270/2539] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:24 2022] DELETE /identity/v3/projects/568f8b139a7f4102a31db8bee764ec2d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:24.694610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:24.707121 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c3e07-5214-420c-8a5d-e2acac3cafea service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:24.769467 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:24.770281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:24.770568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:24.770847 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] PATH_INFO: `/v3/users/6efcb2c1b01b439497551d50b891191c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:24.772033 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efcb2c1b01b439497551d50b891191c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:24.774025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:24.792129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:24.827110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efcb2c1b01b439497551d50b891191c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.841424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efcb2c1b01b439497551d50b891191c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.842745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efcb2c1b01b439497551d50b891191c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.843342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efcb2c1b01b439497551d50b891191c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.855113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efcb2c1b01b439497551d50b891191c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:24.867849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1270/2540] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:24 2022] DELETE /identity/v3/users/6efcb2c1b01b439497551d50b891191c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:25.102330 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.103871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.127009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df83608-9d42-48ea-8042-a53e92451842 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.194957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.199705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.220607 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.221576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.221814 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.222122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] PATH_INFO: `/v3/users/c67205b7150c460a81ff545d5ba29cad` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.223399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67205b7150c460a81ff545d5ba29cad)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.225470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.301752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67205b7150c460a81ff545d5ba29cad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.311479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c704a2-0aac-427d-92b6-2aaea3eaaa50 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.311630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67205b7150c460a81ff545d5ba29cad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.318951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.319082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.319485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.319836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] PATH_INFO: `/v3/projects/5410c932dc2a4781a8e1bff061370e97` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.321228 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5410c932dc2a4781a8e1bff061370e97)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.324337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.328842 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67205b7150c460a81ff545d5ba29cad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.329685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67205b7150c460a81ff545d5ba29cad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.330563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e68256-4737-4b0b-989f-22ebd1375fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67205b7150c460a81ff545d5ba29cad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.347249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1271/2541] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:25 2022] DELETE /identity/v3/users/c67205b7150c460a81ff545d5ba29cad => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:25.385219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5410c932dc2a4781a8e1bff061370e97', 'request_id': 'req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.393043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5410c932dc2a4781a8e1bff061370e97', 'request_id': 'req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.406307 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5410c932dc2a4781a8e1bff061370e97', 'request_id': 'req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.422073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] The token cache is being invalidate because project 5410c932dc2a4781a8e1bff061370e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:25.422365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.423009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e60c0d3c-23d5-48ba-b760-cfecbdd0e256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.424549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1271/2542] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:25 2022] DELETE /identity/v3/projects/5410c932dc2a4781a8e1bff061370e97 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:25.429610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.429905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.430186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.448313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:25.449565 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.500479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:25.516754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1272/2543] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:25.525338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5438-59a1-49b6-a6cf-2422c72dc79b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.525836 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5438-59a1-49b6-a6cf-2422c72dc79b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.526125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5438-59a1-49b6-a6cf-2422c72dc79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.536686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.538164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.549865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a13c5438-59a1-49b6-a6cf-2422c72dc79b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:25.551182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a13c5438-59a1-49b6-a6cf-2422c72dc79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.551382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1190a78-2366-4da4-b000-8fae87b4685e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.556408 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.557329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.557618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.557902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] PATH_INFO: `/v3/projects/7116b23d173c4192a6c8d996ad31d57c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.559983 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7116b23d173c4192a6c8d996ad31d57c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.561615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1272/2544] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:25.566117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c5438-59a1-49b6-a6cf-2422c72dc79b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.567729 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.578237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c5438-59a1-49b6-a6cf-2422c72dc79b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.582989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.583642 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.583926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.584201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.584948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:25.585287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.586999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.604135 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1273/2545] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:25.605125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.611982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:25.616168 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a9746-f5f5-4ba3-bfca-56181a27b3e9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.618627 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7116b23d173c4192a6c8d996ad31d57c', 'request_id': 'req-2017a776-374b-4dcd-94af-59713d9add10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.628321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7116b23d173c4192a6c8d996ad31d57c', 'request_id': 'req-2017a776-374b-4dcd-94af-59713d9add10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.635062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7116b23d173c4192a6c8d996ad31d57c', 'request_id': 'req-2017a776-374b-4dcd-94af-59713d9add10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.646443 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] The token cache is being invalidate because project 7116b23d173c4192a6c8d996ad31d57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:25.646732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.647405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2017a776-374b-4dcd-94af-59713d9add10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:25.649072 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1273/2546] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:25 2022] DELETE /identity/v3/projects/7116b23d173c4192a6c8d996ad31d57c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:25.653750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b97f43-b406-4554-bb55-cccdd83c47dd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.654042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b97f43-b406-4554-bb55-cccdd83c47dd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.654345 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b97f43-b406-4554-bb55-cccdd83c47dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.676735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-60b97f43-b406-4554-bb55-cccdd83c47dd None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:25.677905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60b97f43-b406-4554-bb55-cccdd83c47dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.686763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.687515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.687918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.688206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.688903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.690894 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.719193 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1274/2547] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:25.724132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.726557 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.742298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1274/2548] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:25.750101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73dc7-1738-4ef1-8c32-1aebe3fcac27 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.750393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73dc7-1738-4ef1-8c32-1aebe3fcac27 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.750674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73dc7-1738-4ef1-8c32-1aebe3fcac27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.769066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3ed73dc7-1738-4ef1-8c32-1aebe3fcac27 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:25.770303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ed73dc7-1738-4ef1-8c32-1aebe3fcac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:25.794664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d9247-38a6-47fa-88ed-c9ca4bf1a277 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.800149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.800820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.801103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.801428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.802109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.804384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.827733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1275/2549] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:25.828816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.837542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1275/2550] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:25.839544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed73dc7-1738-4ef1-8c32-1aebe3fcac27 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.841221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37403c73-48ef-4898-9ff4-f8d4cc3b8161 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.846325 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.847182 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.847486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.847798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.848535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.850255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.855379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed73dc7-1738-4ef1-8c32-1aebe3fcac27 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.863989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1276/2551] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:25.864186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.864310 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.864692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.865131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.865665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.866464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:25.866824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.869824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.877485 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac1c30-f6f2-4e51-ba40-1ba7fa5d606f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.883023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.883693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.883979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.884260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.888401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1276/2552] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:25.888572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.888572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.890336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.897186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:25.901404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd47c2c-f68c-423a-8696-a8d884c83efd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.906566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1277/2553] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:25.908024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.908822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.909175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.909477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.910215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.911984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.912236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.922489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbb5dd-736d-4102-9eaf-e96de8b8fdfb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:25.927365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444c090-7323-47be-b9d8-4c5a4ff8c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:25.928334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444c090-7323-47be-b9d8-4c5a4ff8c28f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.928615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444c090-7323-47be-b9d8-4c5a4ff8c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.928888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444c090-7323-47be-b9d8-4c5a4ff8c28f admin admin] PATH_INFO: `/v3/projects/229436f200544e7c96a6099ed408de1c/users/318c821eef154c9b8fb853e572e3fea1/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:25.948153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1277/2554] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:25.948756 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444c090-7323-47be-b9d8-4c5a4ff8c28f admin admin] RBAC: Authorizing `identity:create_grant(project_id=229436f200544e7c96a6099ed408de1c, user_id=318c821eef154c9b8fb853e572e3fea1, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:25.949235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:25.952363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444c090-7323-47be-b9d8-4c5a4ff8c28f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:25.968676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1278/2555] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:25 2022] PUT /identity/v3/projects/229436f200544e7c96a6099ed408de1c/users/318c821eef154c9b8fb853e572e3fea1/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:25.980166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17452ee-a4a6-4166-aa91-570bef371132 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:25.980468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17452ee-a4a6-4166-aa91-570bef371132 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:25.980910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17452ee-a4a6-4166-aa91-570bef371132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.009358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f17452ee-a4a6-4166-aa91-570bef371132 None None] MFA Rules not processed for user `318c821eef154c9b8fb853e572e3fea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:26.010608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f17452ee-a4a6-4166-aa91-570bef371132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:26.017801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787c9b5-2b81-486c-989d-31fc7c71ad77 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.033658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:26.077120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.077772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.078056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.078358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.079049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.080986 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.084989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1279/2556] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:26.089382 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17452ee-a4a6-4166-aa91-570bef371132 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.100564 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17452ee-a4a6-4166-aa91-570bef371132 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.105111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1278/2557] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:25 2022] POST /identity/v3/users => generated 302 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:26.106975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.107832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.108913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.109053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.109355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.110144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.112077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.118579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b7d4d-12c8-4017-a1b1-fe78293c5087 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.124418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.125084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.125394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.125668 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.126402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.127998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.139451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1279/2558] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:26.141910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1280/2559] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:26 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:26.146642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.146818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.156811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e857-de1d-4a84-a6d8-c16bb5b8a27d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.211586 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61abee-c44d-4b1e-84bb-8a2f5a198147 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.216718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.217433 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.217727 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.218032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.218728 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.220679 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.229804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.230478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.230762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.231166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.231901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.234014 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.244918 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1281/2560] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:26 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:26.245135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1280/2561] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:26.246342 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.248946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.256553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:26.258317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ea3dd-79b6-41ce-86bd-dabbe08bf7c2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.262793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034db16-ea2a-47df-b6dd-d131761b48e0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.263552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.264333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.264563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.264837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.265621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.267288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.270437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f666c9e-82a8-4208-8be8-0e688d2da0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.271393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666c9e-82a8-4208-8be8-0e688d2da0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.271674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666c9e-82a8-4208-8be8-0e688d2da0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.271958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666c9e-82a8-4208-8be8-0e688d2da0c8 admin admin] PATH_INFO: `/v3/projects/a1f2e1d51c8a4c3b98ae4eaa8a3968c6/users/40fa9d824b594e3eadff031e4d8192cc/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.281412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1282/2562] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:26.286177 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.293025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f666c9e-82a8-4208-8be8-0e688d2da0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1f2e1d51c8a4c3b98ae4eaa8a3968c6, user_id=40fa9d824b594e3eadff031e4d8192cc, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.297088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f666c9e-82a8-4208-8be8-0e688d2da0c8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.302902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e34231-f654-452e-9320-f28c42a739bb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.307730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.308633 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.309037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.309239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] PATH_INFO: `/v3/projects/ca08500199e14deca4b0abb1e777516f/users/cbb1b24002f54001931e19e7b985e977/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.313235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1281/2563] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:26 2022] PUT /identity/v3/projects/a1f2e1d51c8a4c3b98ae4eaa8a3968c6/users/40fa9d824b594e3eadff031e4d8192cc/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:26.324068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f015d-28b6-4276-9dda-c8b3d51cfd24 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.324516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f015d-28b6-4276-9dda-c8b3d51cfd24 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.326577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f015d-28b6-4276-9dda-c8b3d51cfd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.328067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca08500199e14deca4b0abb1e777516f, user_id=cbb1b24002f54001931e19e7b985e977, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.330909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.345112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1283/2564] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:26 2022] PUT /identity/v3/projects/ca08500199e14deca4b0abb1e777516f/users/cbb1b24002f54001931e19e7b985e977/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:26.349461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.362813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-284f015d-28b6-4276-9dda-c8b3d51cfd24 None None] MFA Rules not processed for user `40fa9d824b594e3eadff031e4d8192cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:26.364610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-284f015d-28b6-4276-9dda-c8b3d51cfd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:26.419220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:26.425249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac856981-9a93-4e81-b99a-a69d6896dbfc admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.431579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.432348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.432874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.433279 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.434674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.436945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.443743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1282/2565] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:26.448006 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1284/2566] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:26.448374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f015d-28b6-4276-9dda-c8b3d51cfd24 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.451885 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.460058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f015d-28b6-4276-9dda-c8b3d51cfd24 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.461702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ad554-8fa9-48b9-a729-993adbaec9c6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.464664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.465449 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.465925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.466208 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.466774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c9054-6f66-4a50-81f5-02f66af4ee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.466934 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.467662 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c9054-6f66-4a50-81f5-02f66af4ee74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.467946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c9054-6f66-4a50-81f5-02f66af4ee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.468222 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c9054-6f66-4a50-81f5-02f66af4ee74 admin admin] PATH_INFO: `/v3/projects/ca08500199e14deca4b0abb1e777516f/users/cbb1b24002f54001931e19e7b985e977/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.468962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.470246 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c9054-6f66-4a50-81f5-02f66af4ee74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca08500199e14deca4b0abb1e777516f, user_id=cbb1b24002f54001931e19e7b985e977, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.472957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c9054-6f66-4a50-81f5-02f66af4ee74 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.486563 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1285/2567] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:26 2022] PUT /identity/v3/projects/ca08500199e14deca4b0abb1e777516f/users/cbb1b24002f54001931e19e7b985e977/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:26.497910 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7b71-fbb3-4122-8df0-186833c56a27 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.498744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7b71-fbb3-4122-8df0-186833c56a27 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.499243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7b71-fbb3-4122-8df0-186833c56a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.500896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1283/2568] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:26 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:26.504659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.528734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-90bf7b71-fbb3-4122-8df0-186833c56a27 None None] MFA Rules not processed for user `cbb1b24002f54001931e19e7b985e977`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:26.530105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-90bf7b71-fbb3-4122-8df0-186833c56a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:26.575604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aacbc-cc7f-4c57-bd4e-505afaee1d80 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.581570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.582271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.582557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.582832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.583536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.585539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.608376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1286/2569] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:26.612184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1284/2570] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:26 2022] POST /identity/v3/users => generated 308 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:26.613292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf7b71-fbb3-4122-8df0-186833c56a27 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.623544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf7b71-fbb3-4122-8df0-186833c56a27 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.623769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.625427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:26.629225 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.629951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.630231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.630507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.631231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.632852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.643389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1287/2571] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:26.647464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.657816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6509a-b58b-45fa-a9f8-bf369071c3f3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.662532 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.663428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.663714 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.663995 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] PATH_INFO: `/v3/projects/ed1667e76ef84d1e88a53309f2406c49/users/3a7dd43807fd405c9bac4e3d91dc45ba/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.677017 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:26.683233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1667e76ef84d1e88a53309f2406c49, user_id=3a7dd43807fd405c9bac4e3d91dc45ba, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.686040 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.694865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f30d-6dc8-4a9b-ba7b-4042a543336d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.698874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.699529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.699816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.700094 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.702455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1288/2572] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:26 2022] PUT /identity/v3/projects/ed1667e76ef84d1e88a53309f2406c49/users/3a7dd43807fd405c9bac4e3d91dc45ba/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:26.706895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.710095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.711579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.721804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1285/2573] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:26.773638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.775188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:26.776446 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe702a-6ef6-4ffd-bc29-3aec1ac58d51 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.781615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.782369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.782630 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.782907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.783643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.785281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.787334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a8d22-9758-48cd-89ce-096054d4bcdf service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.796848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1289/2574] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:26.801131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.813588 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c56d1-4084-476d-96d3-12f00531ad97 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:26.819374 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd0584-fc41-4741-9cc3-8a77f74734cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.820269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0584-fc41-4741-9cc3-8a77f74734cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.820555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0584-fc41-4741-9cc3-8a77f74734cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.820832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0584-fc41-4741-9cc3-8a77f74734cb admin admin] PATH_INFO: `/v3/projects/ed1667e76ef84d1e88a53309f2406c49/users/3a7dd43807fd405c9bac4e3d91dc45ba/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.822868 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd0584-fc41-4741-9cc3-8a77f74734cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1667e76ef84d1e88a53309f2406c49, user_id=3a7dd43807fd405c9bac4e3d91dc45ba, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.825597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd0584-fc41-4741-9cc3-8a77f74734cb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.836538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1290/2575] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:26 2022] PUT /identity/v3/projects/ed1667e76ef84d1e88a53309f2406c49/users/3a7dd43807fd405c9bac4e3d91dc45ba/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:26.840813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:26.841570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.842482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.842780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.850191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186b9f-dfa6-4cbd-9c22-4269c76c8f27 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:26.850572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186b9f-dfa6-4cbd-9c22-4269c76c8f27 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:26.850882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186b9f-dfa6-4cbd-9c22-4269c76c8f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:26.874255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:26.883202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a1186b9f-dfa6-4cbd-9c22-4269c76c8f27 None None] MFA Rules not processed for user `3a7dd43807fd405c9bac4e3d91dc45ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:26.885058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a1186b9f-dfa6-4cbd-9c22-4269c76c8f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:26.905326 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:26.906784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:26.916731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1286/2576] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:26.964929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1291/2577] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:26.976683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:26.986008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96d5b5-0b88-4799-b3fd-aff894545133 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:27.040130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34962-8a66-4765-ba58-a26a5eb7c9ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:27.040866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34962-8a66-4765-ba58-a26a5eb7c9ba service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:27.041172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34962-8a66-4765-ba58-a26a5eb7c9ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:27.041462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34962-8a66-4765-ba58-a26a5eb7c9ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:27.052378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34962-8a66-4765-ba58-a26a5eb7c9ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:27.053935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34962-8a66-4765-ba58-a26a5eb7c9ba service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:27.064417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1287/2578] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:27.243864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186b9f-dfa6-4cbd-9c22-4269c76c8f27 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:27.312976 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186b9f-dfa6-4cbd-9c22-4269c76c8f27 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:27.317610 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:27.318526 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:27.318806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:27.319105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] PATH_INFO: `/v3/users/40fa9d824b594e3eadff031e4d8192cc` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:27.320309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40fa9d824b594e3eadff031e4d8192cc)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:27.322299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:27.372964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40fa9d824b594e3eadff031e4d8192cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.387212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40fa9d824b594e3eadff031e4d8192cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.387937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40fa9d824b594e3eadff031e4d8192cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.388554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40fa9d824b594e3eadff031e4d8192cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.397541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40fa9d824b594e3eadff031e4d8192cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.412082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1292/2579] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:27 2022] DELETE /identity/v3/users/40fa9d824b594e3eadff031e4d8192cc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:27.419815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:27.419815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:27.419815 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:27.436364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:27.437648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:27.472556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:27.509288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1288/2580] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:27.519963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:27.521487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:27.540790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:27.588308 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104369f2-2bcb-45a6-8977-93a907f0b516 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:27.592011 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:27.592663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:27.592945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:27.593246 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:27.603226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:27.604674 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:27.614871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1293/2581] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:27.896862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:27.898364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:27.911145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ab717-652d-4b9c-b068-b213d65d9888 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:27.916669 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:27.917583 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:27.917876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:27.918165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] PATH_INFO: `/v3/projects/a1f2e1d51c8a4c3b98ae4eaa8a3968c6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:27.919366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1f2e1d51c8a4c3b98ae4eaa8a3968c6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:27.921945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:27.969333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1f2e1d51c8a4c3b98ae4eaa8a3968c6', 'request_id': 'req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.976943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1f2e1d51c8a4c3b98ae4eaa8a3968c6', 'request_id': 'req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.983735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1f2e1d51c8a4c3b98ae4eaa8a3968c6', 'request_id': 'req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.994303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] The token cache is being invalidate because project a1f2e1d51c8a4c3b98ae4eaa8a3968c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:27.994588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.995223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:27.996756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1289/2582] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:27 2022] DELETE /identity/v3/projects/a1f2e1d51c8a4c3b98ae4eaa8a3968c6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:28.001527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:28.011144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd2617-d327-4976-9e31-6ecce3424c55 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:28.066488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:28.067359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:28.067639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:28.067909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] PATH_INFO: `/v3/users/3a7dd43807fd405c9bac4e3d91dc45ba` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:28.069074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7dd43807fd405c9bac4e3d91dc45ba)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:28.071032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:28.121109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7dd43807fd405c9bac4e3d91dc45ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.132932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7dd43807fd405c9bac4e3d91dc45ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.133628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7dd43807fd405c9bac4e3d91dc45ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.134232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7dd43807fd405c9bac4e3d91dc45ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.141989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-596f565a-3caf-4de1-961b-efead8a5223c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7dd43807fd405c9bac4e3d91dc45ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.153977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1294/2583] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:28 2022] DELETE /identity/v3/users/3a7dd43807fd405c9bac4e3d91dc45ba => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:28.434166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:28.435756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:28.448652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daf89e-e913-4564-9ed0-4cf22d64ffeb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:28.512992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:28.514477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:28.514847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:28.515339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] PATH_INFO: `/v3/projects/ed1667e76ef84d1e88a53309f2406c49` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:28.516447 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1667e76ef84d1e88a53309f2406c49)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:28.518404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:28.573686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1667e76ef84d1e88a53309f2406c49', 'request_id': 'req-6c6c0a08-672b-4d62-aa17-38338e956afa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.581693 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1667e76ef84d1e88a53309f2406c49', 'request_id': 'req-6c6c0a08-672b-4d62-aa17-38338e956afa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.589871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1667e76ef84d1e88a53309f2406c49', 'request_id': 'req-6c6c0a08-672b-4d62-aa17-38338e956afa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.592554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:28.605166 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] The token cache is being invalidate because project ed1667e76ef84d1e88a53309f2406c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:28.605491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.606311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6c6c0a08-672b-4d62-aa17-38338e956afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:28.608276 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1290/2584] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:28 2022] DELETE /identity/v3/projects/ed1667e76ef84d1e88a53309f2406c49 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:28.613385 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d33764-fd2a-4b9d-9d61-8f28330fde97 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:28.613888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d33764-fd2a-4b9d-9d61-8f28330fde97 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:28.614004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d33764-fd2a-4b9d-9d61-8f28330fde97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:28.643540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-74d33764-fd2a-4b9d-9d61-8f28330fde97 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:28.644678 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-74d33764-fd2a-4b9d-9d61-8f28330fde97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:28.682132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:28.729372 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1295/2585] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:28.741521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918dae0-e0fe-4ba5-ab60-bfb63ecbc842 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:28.742007 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918dae0-e0fe-4ba5-ab60-bfb63ecbc842 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:28.742489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918dae0-e0fe-4ba5-ab60-bfb63ecbc842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:28.763238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d918dae0-e0fe-4ba5-ab60-bfb63ecbc842 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:28.765063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d918dae0-e0fe-4ba5-ab60-bfb63ecbc842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:28.774393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1291/2586] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:28.778677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d33764-fd2a-4b9d-9d61-8f28330fde97 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:28.789542 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d33764-fd2a-4b9d-9d61-8f28330fde97 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:28.794178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:28.794870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:28.795159 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:28.795444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:28.796170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:28.796477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:28.798142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:28.811431 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1296/2587] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:28.817460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918dae0-e0fe-4ba5-ab60-bfb63ecbc842 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:28.827742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918dae0-e0fe-4ba5-ab60-bfb63ecbc842 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:28.832366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:28.833030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:28.833334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:28.833611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:28.834299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:28.836199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:28.863804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1292/2588] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:28 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:28.868047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:28.879314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e54da5-30fa-4cd9-904f-b4a1028ee1c0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:28.952908 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:28.953773 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:28.954067 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:28.954346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:28.955095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:28.957062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:28.983162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1297/2589] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:28 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:28.984340 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:28.994593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e7bd2-fe4a-4777-beb0-d002c101bffc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:28.999637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.000294 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.000575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.000850 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.001689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.003271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.014363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1293/2590] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:29.018977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.028863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b77f6-0a12-4c1f-9725-5ce56bdf4fad admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.033371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4ca0d-63fa-40a9-92d7-5732c57b3ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.034022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4ca0d-63fa-40a9-92d7-5732c57b3ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.034301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4ca0d-63fa-40a9-92d7-5732c57b3ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.034577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4ca0d-63fa-40a9-92d7-5732c57b3ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.035289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4ca0d-63fa-40a9-92d7-5732c57b3ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.036891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4ca0d-63fa-40a9-92d7-5732c57b3ed4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.047088 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1298/2591] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:29.052312 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.063943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457913-9f40-4060-b074-010ea6357545 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.069295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.070192 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.070477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.070747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] PATH_INFO: `/v3/projects/820e70ea33fa4564bdfeaaef078c413d/users/fbf6fe88457f45bba3c1c4253258ace1/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.086804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820e70ea33fa4564bdfeaaef078c413d, user_id=fbf6fe88457f45bba3c1c4253258ace1, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.089572 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.104303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1294/2592] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:29 2022] PUT /identity/v3/projects/820e70ea33fa4564bdfeaaef078c413d/users/fbf6fe88457f45bba3c1c4253258ace1/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:29.115211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1bc37-0b3b-451c-9515-b30f232ebdde None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.115547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1bc37-0b3b-451c-9515-b30f232ebdde None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.115839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1bc37-0b3b-451c-9515-b30f232ebdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.145999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-11e1bc37-0b3b-451c-9515-b30f232ebdde None None] MFA Rules not processed for user `fbf6fe88457f45bba3c1c4253258ace1`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:29.147198 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-11e1bc37-0b3b-451c-9515-b30f232ebdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:29.228786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1299/2593] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:29.233658 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.300883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4df71c-c0e5-49e3-9d04-db43d54ed2c7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.305898 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.306584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.306873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.307152 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.307862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.309824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.340517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1295/2594] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:29 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:29.345337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1bc37-0b3b-451c-9515-b30f232ebdde None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.358369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1bc37-0b3b-451c-9515-b30f232ebdde None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.381281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:29.421912 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.422571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.422859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.423142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.423841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.425820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.444873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1300/2595] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:29 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:29.448890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.459803 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688c7fe-9482-4b85-89c4-5cb0f83c3c96 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.464429 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.465090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.465409 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.465692 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.466420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.468695 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.479998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1296/2596] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:29.484169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.494196 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7bc6a-4257-4415-8516-a2e270683777 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.498865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.499743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.500026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.500304 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] PATH_INFO: `/v3/projects/fa58ccfb636b4935be7ff3520c4cd2f4/users/2d334b0df86d4b93a2aa579a897f14cc/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.517959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa58ccfb636b4935be7ff3520c4cd2f4, user_id=2d334b0df86d4b93a2aa579a897f14cc, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.520634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.533440 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1301/2597] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:29 2022] PUT /identity/v3/projects/fa58ccfb636b4935be7ff3520c4cd2f4/users/2d334b0df86d4b93a2aa579a897f14cc/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:29.537382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.547265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300122f9-1cb9-440e-bec8-c17ac635068d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.604325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c4d9b-4016-4790-b4fe-b1b53ccc3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.604992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c4d9b-4016-4790-b4fe-b1b53ccc3375 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.605311 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c4d9b-4016-4790-b4fe-b1b53ccc3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.605591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c4d9b-4016-4790-b4fe-b1b53ccc3375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.606318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c4d9b-4016-4790-b4fe-b1b53ccc3375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.607925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c4d9b-4016-4790-b4fe-b1b53ccc3375 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.610112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:29.622091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1297/2598] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:29.623739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.633919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295be76f-7d25-4a51-8664-6fba7cc7b2da admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.639327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.640214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.640499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.640778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] PATH_INFO: `/v3/projects/fa58ccfb636b4935be7ff3520c4cd2f4/users/2d334b0df86d4b93a2aa579a897f14cc/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.642855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa58ccfb636b4935be7ff3520c4cd2f4, user_id=2d334b0df86d4b93a2aa579a897f14cc, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.645527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.655729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1302/2599] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:29 2022] PUT /identity/v3/projects/fa58ccfb636b4935be7ff3520c4cd2f4/users/2d334b0df86d4b93a2aa579a897f14cc/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:29.666927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5064a-70b3-4fe4-94ad-b6f4982670b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.667227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5064a-70b3-4fe4-94ad-b6f4982670b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.667511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5064a-70b3-4fe4-94ad-b6f4982670b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.693621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b7d5064a-70b3-4fe4-94ad-b6f4982670b3 None None] MFA Rules not processed for user `2d334b0df86d4b93a2aa579a897f14cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:29.694825 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7d5064a-70b3-4fe4-94ad-b6f4982670b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:29.768563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1298/2600] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:29.779725 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:29.784709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:29.848426 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a2c67-14ed-40c0-acc7-11f5264cfd33 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:29.853283 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b896ca2-249d-45dc-a8e2-07889b2a1f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:29.854230 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b896ca2-249d-45dc-a8e2-07889b2a1f37 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:29.854353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b896ca2-249d-45dc-a8e2-07889b2a1f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:29.854690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b896ca2-249d-45dc-a8e2-07889b2a1f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:29.865533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b896ca2-249d-45dc-a8e2-07889b2a1f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:29.867142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b896ca2-249d-45dc-a8e2-07889b2a1f37 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:29.877084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1303/2601] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:29.953538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5064a-70b3-4fe4-94ad-b6f4982670b3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:30.031830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5064a-70b3-4fe4-94ad-b6f4982670b3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:30.036933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:30.037853 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.038130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.038403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] PATH_INFO: `/v3/users/fbf6fe88457f45bba3c1c4253258ace1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.039615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbf6fe88457f45bba3c1c4253258ace1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:30.041615 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:30.106568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbf6fe88457f45bba3c1c4253258ace1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.117857 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbf6fe88457f45bba3c1c4253258ace1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.133197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbf6fe88457f45bba3c1c4253258ace1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.134088 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbf6fe88457f45bba3c1c4253258ace1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.134994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbf6fe88457f45bba3c1c4253258ace1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.156198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1299/2602] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:29 2022] DELETE /identity/v3/users/fbf6fe88457f45bba3c1c4253258ace1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:30.162296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.162595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.163161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.187111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:30.188471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:30.257380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1304/2603] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:30.268781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:30.270386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:30.275723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:30.339989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:30.341170 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcb834-4fc5-43c8-b78d-acc4aa568ff3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:30.341533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:30.345468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12272f5-802e-40a8-b106-54999cfb07b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:30.346121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12272f5-802e-40a8-b106-54999cfb07b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.346406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12272f5-802e-40a8-b106-54999cfb07b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.346682 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12272f5-802e-40a8-b106-54999cfb07b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.354330 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980d1d4-f358-474c-b6bd-1732b5989dcc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:30.357709 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12272f5-802e-40a8-b106-54999cfb07b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:30.359250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12272f5-802e-40a8-b106-54999cfb07b5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:30.361750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:30.362710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.362992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.363274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] PATH_INFO: `/v3/users/318c821eef154c9b8fb853e572e3fea1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.364487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=318c821eef154c9b8fb853e572e3fea1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:30.366458 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:30.371559 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1300/2604] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:30.382620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:30.426253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318c821eef154c9b8fb853e572e3fea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.443448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318c821eef154c9b8fb853e572e3fea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.444255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318c821eef154c9b8fb853e572e3fea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.445251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318c821eef154c9b8fb853e572e3fea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.455186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318c821eef154c9b8fb853e572e3fea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.471683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1305/2605] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:30 2022] DELETE /identity/v3/users/318c821eef154c9b8fb853e572e3fea1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:30.476126 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.476648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.476774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.498510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:30.499705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:30.580673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1301/2606] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:30.590495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:30.591987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:30.658132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401cc59-e730-4bcf-8681-bfaeb52b8603 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:30.662843 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:30.663641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.664020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.664391 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.680890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:30.680890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:30.682344 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:30.684164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:30.698257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1306/2607] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:30.705145 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f9084-e2d1-407c-b752-60b22a690b28 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:30.711175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:30.715328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.716117 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.716722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] PATH_INFO: `/v3/projects/820e70ea33fa4564bdfeaaef078c413d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.719561 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=820e70ea33fa4564bdfeaaef078c413d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:30.721806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:30.779098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820e70ea33fa4564bdfeaaef078c413d', 'request_id': 'req-85acf0c1-1158-4a0b-9411-c409ec1dba06'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.786936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820e70ea33fa4564bdfeaaef078c413d', 'request_id': 'req-85acf0c1-1158-4a0b-9411-c409ec1dba06'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.794402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820e70ea33fa4564bdfeaaef078c413d', 'request_id': 'req-85acf0c1-1158-4a0b-9411-c409ec1dba06'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.810155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] The token cache is being invalidate because project 820e70ea33fa4564bdfeaaef078c413d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:30.811061 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.811762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.813378 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1302/2608] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:30 2022] DELETE /identity/v3/projects/820e70ea33fa4564bdfeaaef078c413d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:30.817848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:30.828863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c724d59-df7a-4e2c-b616-0c2ff4d6b5f9 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:30.904705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:30.908651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:30.908651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:30.908651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] PATH_INFO: `/v3/users/2d334b0df86d4b93a2aa579a897f14cc` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:30.908651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d334b0df86d4b93a2aa579a897f14cc)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:30.910154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:30.964402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:30.976607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d334b0df86d4b93a2aa579a897f14cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.989406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:30.990987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:30.994016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d334b0df86d4b93a2aa579a897f14cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.994747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d334b0df86d4b93a2aa579a897f14cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:30.995501 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d334b0df86d4b93a2aa579a897f14cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.006609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d334b0df86d4b93a2aa579a897f14cc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.018095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acf0c1-1158-4a0b-9411-c409ec1dba06 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.023478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.024360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.024641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.024918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] PATH_INFO: `/v3/projects/229436f200544e7c96a6099ed408de1c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.025037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1307/2609] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:30 2022] DELETE /identity/v3/users/2d334b0df86d4b93a2aa579a897f14cc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:31.027985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=229436f200544e7c96a6099ed408de1c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.028320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.057441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:31.086247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229436f200544e7c96a6099ed408de1c', 'request_id': 'req-c6c82c39-ab0e-4480-ad82-9be17486f7bc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.096271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229436f200544e7c96a6099ed408de1c', 'request_id': 'req-c6c82c39-ab0e-4480-ad82-9be17486f7bc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.103800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229436f200544e7c96a6099ed408de1c', 'request_id': 'req-c6c82c39-ab0e-4480-ad82-9be17486f7bc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.122135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] The token cache is being invalidate because project 229436f200544e7c96a6099ed408de1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:31.122864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.123662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.126151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1303/2610] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:30 2022] DELETE /identity/v3/projects/229436f200544e7c96a6099ed408de1c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:31.130346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.131861 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.217173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9dc20-1a83-4c30-be29-f76ec2d52c8b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.301471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.302380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.302665 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.303140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] PATH_INFO: `/v3/users/cbb1b24002f54001931e19e7b985e977` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.304393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb1b24002f54001931e19e7b985e977)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.306513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.361297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb1b24002f54001931e19e7b985e977'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.374507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.375301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:31.375978 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.377893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb1b24002f54001931e19e7b985e977'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.378883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb1b24002f54001931e19e7b985e977'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.380137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb1b24002f54001931e19e7b985e977'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.393941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c82c39-ab0e-4480-ad82-9be17486f7bc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.396599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f67a4262-ecfb-4f9a-aa80-51ea7e2befdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb1b24002f54001931e19e7b985e977'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.402417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.408595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.409228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.410585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] PATH_INFO: `/v3/projects/fa58ccfb636b4935be7ff3520c4cd2f4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.412373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa58ccfb636b4935be7ff3520c4cd2f4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.415043 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.419884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1308/2611] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:31 2022] DELETE /identity/v3/users/cbb1b24002f54001931e19e7b985e977 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:31.473079 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa58ccfb636b4935be7ff3520c4cd2f4', 'request_id': 'req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.482505 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa58ccfb636b4935be7ff3520c4cd2f4', 'request_id': 'req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.490038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa58ccfb636b4935be7ff3520c4cd2f4', 'request_id': 'req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.500862 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] The token cache is being invalidate because project fa58ccfb636b4935be7ff3520c4cd2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:31.501181 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.501845 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ad6e7bf-faf2-4bce-8098-ddfb2f860bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.503395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1304/2612] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:31 2022] DELETE /identity/v3/projects/fa58ccfb636b4935be7ff3520c4cd2f4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:31.509208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c60dd-d7bb-47a1-8192-fbc0dfa94370 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.509503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c60dd-d7bb-47a1-8192-fbc0dfa94370 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.509787 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c60dd-d7bb-47a1-8192-fbc0dfa94370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.531946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d94c60dd-d7bb-47a1-8192-fbc0dfa94370 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:31.533215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d94c60dd-d7bb-47a1-8192-fbc0dfa94370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.611357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1309/2613] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:31.619120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcfd4d-482a-43f0-8d86-3a9078713f8f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.619410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcfd4d-482a-43f0-8d86-3a9078713f8f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.619681 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcfd4d-482a-43f0-8d86-3a9078713f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.640236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-65fcfd4d-482a-43f0-8d86-3a9078713f8f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:31.641613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-65fcfd4d-482a-43f0-8d86-3a9078713f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.650528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1305/2614] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:31.654905 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c60dd-d7bb-47a1-8192-fbc0dfa94370 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.667638 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c60dd-d7bb-47a1-8192-fbc0dfa94370 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.674170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.675188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.675658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.676117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.677176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:31.677629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.680320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.694052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1310/2615] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:08:31.698298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcfd4d-482a-43f0-8d86-3a9078713f8f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.708923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcfd4d-482a-43f0-8d86-3a9078713f8f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.713958 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.714626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.714948 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.715176 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.715852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.718444 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.725299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.727704 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.743545 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ff591-2e23-44af-afda-8c54f7db2b96 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.748661 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.750099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.750589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.751151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] PATH_INFO: `/v3/projects/ca08500199e14deca4b0abb1e777516f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.752182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1306/2616] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:31 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:31.754112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca08500199e14deca4b0abb1e777516f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.756194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.756413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.806207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca08500199e14deca4b0abb1e777516f', 'request_id': 'req-d760fd40-5699-4717-8cfd-36b788151d64'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.814320 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca08500199e14deca4b0abb1e777516f', 'request_id': 'req-d760fd40-5699-4717-8cfd-36b788151d64'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.820707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca08500199e14deca4b0abb1e777516f', 'request_id': 'req-d760fd40-5699-4717-8cfd-36b788151d64'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.821260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbdcaa-c417-41d3-aee5-bbf28c7a9e4a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.831594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:31.834620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] The token cache is being invalidate because project ca08500199e14deca4b0abb1e777516f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:31.834913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.835571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d760fd40-5699-4717-8cfd-36b788151d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:31.837170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1311/2617] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:31 2022] DELETE /identity/v3/projects/ca08500199e14deca4b0abb1e777516f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:31.841750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d57c3-c3a9-4f55-a52c-aff4b37fdc3a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.842129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d57c3-c3a9-4f55-a52c-aff4b37fdc3a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.842546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d57c3-c3a9-4f55-a52c-aff4b37fdc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.860860 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-075d57c3-c3a9-4f55-a52c-aff4b37fdc3a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:31.862090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-075d57c3-c3a9-4f55-a52c-aff4b37fdc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.887547 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.889381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.889689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.889974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.890679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.892624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.909188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:31.916203 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1307/2618] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:31 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:31.917253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.919086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.925810 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1312/2619] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:31.930542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb5080-9723-4d1e-a816-a914b994cdd4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.932968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837e141-019b-4c29-b4fa-10c98fa02e46 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.933357 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837e141-019b-4c29-b4fa-10c98fa02e46 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.933647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837e141-019b-4c29-b4fa-10c98fa02e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.936059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.936716 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.936996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.937306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.938023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.939614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.953300 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1308/2620] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:31.953510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f837e141-019b-4c29-b4fa-10c98fa02e46 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:31.954347 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.954535 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f837e141-019b-4c29-b4fa-10c98fa02e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:31.962370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1313/2621] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:31.964100 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bf725-5f40-4ea3-a86a-d1e54cb111fe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.966247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837e141-019b-4c29-b4fa-10c98fa02e46 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:31.969168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.969831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.970118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.970594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.971319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.973697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.975889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837e141-019b-4c29-b4fa-10c98fa02e46 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:31.980755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:31.981437 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:31.981723 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:31.981999 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:31.982755 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:31.983059 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:31.987617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1309/2622] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:31.987785 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:31.988618 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.000968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1314/2623] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:08:32.001134 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c4af9-f849-4bb6-bb6f-5f249aae21ca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.001901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.004101 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d636-156e-4730-b599-900dbfa54abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.004980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d636-156e-4730-b599-900dbfa54abf admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.005296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d636-156e-4730-b599-900dbfa54abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.005573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d636-156e-4730-b599-900dbfa54abf admin admin] PATH_INFO: `/v3/projects/bdf99ab130464dd0a4babe0da87c0fde/users/dc6be2a4a55147dcb864e44b82f24ed8/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.012898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9d158-0104-491a-8b14-e25f752f5469 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.018197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.018848 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.019233 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.019530 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.020210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.022141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d636-156e-4730-b599-900dbfa54abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf99ab130464dd0a4babe0da87c0fde, user_id=dc6be2a4a55147dcb864e44b82f24ed8, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.022265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.024793 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d636-156e-4730-b599-900dbfa54abf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.038712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1310/2624] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:31 2022] PUT /identity/v3/projects/bdf99ab130464dd0a4babe0da87c0fde/users/dc6be2a4a55147dcb864e44b82f24ed8/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:32.048871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1315/2625] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:32.049038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae82707-72a3-430d-8368-f830beda9949 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.049182 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae82707-72a3-430d-8368-f830beda9949 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.049370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae82707-72a3-430d-8368-f830beda9949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.052543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.080526 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8ae82707-72a3-430d-8368-f830beda9949 None None] MFA Rules not processed for user `dc6be2a4a55147dcb864e44b82f24ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:32.081691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8ae82707-72a3-430d-8368-f830beda9949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:32.121370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a502c-5024-4755-a822-ce7a6a07df1f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.126746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.127422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.127707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.127985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.128685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.130647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.150510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1311/2626] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:32.154173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1316/2627] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:32.154957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.155380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae82707-72a3-430d-8368-f830beda9949 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.166883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e141e-4f49-4c02-a9ff-595848f7304c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.168263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae82707-72a3-430d-8368-f830beda9949 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.172118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.172782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.173070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.173200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.173386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.177483 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.177619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.177619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.177619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.177619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.177971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.178070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.190524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1312/2628] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:32.194413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.205140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf211f-d69c-4abb-b247-8e847359e9dd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.205495 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1317/2629] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:32.209496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.239293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:32.265728 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.266401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.266686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.267103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.267835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.269451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.273144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d5d27-bbc1-4f4f-9353-6568b300aee4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.277803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.278469 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.278754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.279035 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.279425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1313/2630] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:32.281596 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.283537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.283660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.293229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbda68-e894-4bb6-ae6a-8c96de74b5e6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.299468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3459a01-b2bd-401a-8576-23e0f55b8b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.300754 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3459a01-b2bd-401a-8576-23e0f55b8b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.301426 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3459a01-b2bd-401a-8576-23e0f55b8b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.301719 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3459a01-b2bd-401a-8576-23e0f55b8b9a admin admin] PATH_INFO: `/v3/projects/d2f48acb82ea4164835c401f9dffdcc3/users/6b6355d5e0bd43678e7768663d66f8d4/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.309375 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1318/2631] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:32.310444 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.318235 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3459a01-b2bd-401a-8576-23e0f55b8b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f48acb82ea4164835c401f9dffdcc3, user_id=6b6355d5e0bd43678e7768663d66f8d4, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.320811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ee42-8b5a-4622-b85e-ee1ca4749a1b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.321305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3459a01-b2bd-401a-8576-23e0f55b8b9a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.326922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.328092 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.328457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.328745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.329803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.331488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.332887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:32.336155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1314/2632] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:32 2022] PUT /identity/v3/projects/d2f48acb82ea4164835c401f9dffdcc3/users/6b6355d5e0bd43678e7768663d66f8d4/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:32.344409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1319/2633] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:32.346401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56238-5728-45a2-b1ba-b9a9f7f87f52 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.346694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56238-5728-45a2-b1ba-b9a9f7f87f52 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.346974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56238-5728-45a2-b1ba-b9a9f7f87f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.347960 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.357937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cfa52-22c1-4071-b6a2-c5df7deec0af admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.376393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5bb56238-5728-45a2-b1ba-b9a9f7f87f52 None None] MFA Rules not processed for user `6b6355d5e0bd43678e7768663d66f8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:32.377614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5bb56238-5728-45a2-b1ba-b9a9f7f87f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:32.416958 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.417889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.418183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.418461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] PATH_INFO: `/v3/projects/98a045e9fa814cdbaa5d1dcb41f4b0de/users/f4d7337d63194363aefefc158ee1ba0a/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.434189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a045e9fa814cdbaa5d1dcb41f4b0de, user_id=f4d7337d63194363aefefc158ee1ba0a, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.437606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.443809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1315/2634] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:32.448335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56238-5728-45a2-b1ba-b9a9f7f87f52 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.452009 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1320/2635] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:32 2022] PUT /identity/v3/projects/98a045e9fa814cdbaa5d1dcb41f4b0de/users/f4d7337d63194363aefefc158ee1ba0a/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:32.455595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.458838 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56238-5728-45a2-b1ba-b9a9f7f87f52 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.517369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.518068 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.521438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.521438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.521667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c7580-db1c-4a12-ab9b-73f3556d8ea0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.521774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.521774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.524132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.524805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.525218 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.525411 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.526095 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.527819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.541199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1321/2636] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:32.542790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.551942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1316/2637] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:32.552831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.553466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993014c5-8b93-479e-be22-ffe31d7055fb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.577890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:32.613034 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480650c7-e421-4ec8-b419-058a212a6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.613904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480650c7-e421-4ec8-b419-058a212a6a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.614188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480650c7-e421-4ec8-b419-058a212a6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.614466 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480650c7-e421-4ec8-b419-058a212a6a07 admin admin] PATH_INFO: `/v3/projects/98a045e9fa814cdbaa5d1dcb41f4b0de/users/f4d7337d63194363aefefc158ee1ba0a/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.616477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480650c7-e421-4ec8-b419-058a212a6a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a045e9fa814cdbaa5d1dcb41f4b0de, user_id=f4d7337d63194363aefefc158ee1ba0a, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.617251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b366ae-4cb8-4f4d-bcd9-7dc695451b56 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.619561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480650c7-e421-4ec8-b419-058a212a6a07 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.622143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.622811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.623124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.623399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.624120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.626070 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.630244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1322/2638] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:32 2022] PUT /identity/v3/projects/98a045e9fa814cdbaa5d1dcb41f4b0de/users/f4d7337d63194363aefefc158ee1ba0a/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:32.642049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.642323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.642603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.649301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1317/2639] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:32.651027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.661425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53368ff8-6ec6-437b-8bf8-98d84c19d24d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.662166 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:32.672475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] MFA Rules not processed for user `f4d7337d63194363aefefc158ee1ba0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:32.673932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:32.726295 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.726974 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.727250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.727529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.728255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.729888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.741238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1318/2640] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:32.745022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.753605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1323/2641] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:32.755528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02ac72-2bd5-415b-94ef-c673da10c8d9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.760562 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.761731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.762013 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.762289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] PATH_INFO: `/v3/projects/e833517b53454447915a423c49167866/users/ce9e76de90004ff5abb77026b4e59be3/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.764739 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.766150 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:32.777599 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e833517b53454447915a423c49167866, user_id=ce9e76de90004ff5abb77026b4e59be3, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.780851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.794169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1319/2642] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:32 2022] PUT /identity/v3/projects/e833517b53454447915a423c49167866/users/ce9e76de90004ff5abb77026b4e59be3/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:32.798710 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.828329 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f16e4-a1c4-4ddc-80d2-92040e5f5b9b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.865567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf882e-1311-4222-861f-e17b74581591 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.870341 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.871009 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.871309 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.871577 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.872297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.874112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.879334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.879989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.880276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.880558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.883917 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1320/2643] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:32.888501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:32.897866 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23899a3b-2323-4758-971b-126cb39b2856 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:32.902528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714cf13-f98b-409a-8f2c-ba17210b73bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:32.903404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714cf13-f98b-409a-8f2c-ba17210b73bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.903690 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714cf13-f98b-409a-8f2c-ba17210b73bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.903970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714cf13-f98b-409a-8f2c-ba17210b73bc admin admin] PATH_INFO: `/v3/projects/e833517b53454447915a423c49167866/users/ce9e76de90004ff5abb77026b4e59be3/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.906829 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714cf13-f98b-409a-8f2c-ba17210b73bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e833517b53454447915a423c49167866, user_id=ce9e76de90004ff5abb77026b4e59be3, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.909534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714cf13-f98b-409a-8f2c-ba17210b73bc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.920760 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1321/2644] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:32 2022] PUT /identity/v3/projects/e833517b53454447915a423c49167866/users/ce9e76de90004ff5abb77026b4e59be3/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:32.931677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:32.932077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:32.932370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:32.941875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:32.943324 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:32.945074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:32.948533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:32.954272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1324/2645] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:32 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:32.961633 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] MFA Rules not processed for user `ce9e76de90004ff5abb77026b4e59be3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:32.962865 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:33.037840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:33.043521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1322/2646] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:33.047226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ed5fe-d6e2-4fe6-9d84-51a4bcec63f7 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:33.056691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:33.058318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:33.102830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a7546-f3cd-4b0f-abf0-2df53d5993a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:33.103802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a7546-f3cd-4b0f-abf0-2df53d5993a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:33.103802 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a7546-f3cd-4b0f-abf0-2df53d5993a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:33.104037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a7546-f3cd-4b0f-abf0-2df53d5993a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:33.123686 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7a07c-7001-4096-ae68-8c6c9413e114 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:33.127783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:33.128433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:33.128715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:33.128992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:33.139277 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:33.141533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:33.151118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1323/2647] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:33.164204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a7546-f3cd-4b0f-abf0-2df53d5993a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:33.165686 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a7546-f3cd-4b0f-abf0-2df53d5993a3 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:33.175846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1325/2648] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:33.181249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:33.258273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37f4bc-4466-492f-a25f-901bb789bd82 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:33.263212 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:33.264621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:33.264903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:33.265503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] PATH_INFO: `/v3/users/6b6355d5e0bd43678e7768663d66f8d4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:33.267197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6355d5e0bd43678e7768663d66f8d4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:33.269301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:33.314213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:33.325022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6355d5e0bd43678e7768663d66f8d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.335053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6355d5e0bd43678e7768663d66f8d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.351143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6355d5e0bd43678e7768663d66f8d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.351950 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6355d5e0bd43678e7768663d66f8d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.352573 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6355d5e0bd43678e7768663d66f8d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.365702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1324/2649] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:33 2022] DELETE /identity/v3/users/6b6355d5e0bd43678e7768663d66f8d4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:33.370262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:33.370566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:33.370903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:33.391474 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:33.392641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:33.477412 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1326/2650] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:33.488265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:33.489868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:33.554023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eb62e-eed9-45fa-b7da-2d631a55f1e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:33.557747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:33.558397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:33.558685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:33.558962 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:33.569543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:33.570976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:33.581318 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1325/2651] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:33.880214 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:33.881771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:33.896348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2a234-5818-4c3e-9087-5210a6524401 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:33.912853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:33.914016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:33.915085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:33.915368 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] PATH_INFO: `/v3/projects/d2f48acb82ea4164835c401f9dffdcc3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:33.916781 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f48acb82ea4164835c401f9dffdcc3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:33.918865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:33.970593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f48acb82ea4164835c401f9dffdcc3', 'request_id': 'req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.971790 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:33.979397 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f48acb82ea4164835c401f9dffdcc3', 'request_id': 'req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.985813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f48acb82ea4164835c401f9dffdcc3', 'request_id': 'req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:33.999677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] The token cache is being invalidate because project d2f48acb82ea4164835c401f9dffdcc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:33.999965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.000589 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.002163 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1327/2652] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:33 2022] DELETE /identity/v3/projects/d2f48acb82ea4164835c401f9dffdcc3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:34.006141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:34.015970 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8930b-1ba7-4c7d-a2c9-b2de2f278ec2 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:34.070995 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:34.071867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.072150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.072424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] PATH_INFO: `/v3/users/ce9e76de90004ff5abb77026b4e59be3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.073636 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9e76de90004ff5abb77026b4e59be3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:34.075536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:34.122221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9e76de90004ff5abb77026b4e59be3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.129859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9e76de90004ff5abb77026b4e59be3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.134354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:34.144597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9e76de90004ff5abb77026b4e59be3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.145287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9e76de90004ff5abb77026b4e59be3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.145894 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-79d4b8b0-07ab-404a-8e34-36b9b0956ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9e76de90004ff5abb77026b4e59be3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.158477 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1326/2653] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:34 2022] DELETE /identity/v3/users/ce9e76de90004ff5abb77026b4e59be3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:34.431137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:34.432856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:34.447573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd7ae7-b89b-462f-9602-ec75ba50ed9d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:34.506468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:34.507354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.507649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.507916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] PATH_INFO: `/v3/projects/e833517b53454447915a423c49167866` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.509114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e833517b53454447915a423c49167866)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:34.511388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:34.588443 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e833517b53454447915a423c49167866', 'request_id': 'req-1451c7d8-6a5f-4d30-a24d-076da7182629'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.596519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e833517b53454447915a423c49167866', 'request_id': 'req-1451c7d8-6a5f-4d30-a24d-076da7182629'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.603280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e833517b53454447915a423c49167866', 'request_id': 'req-1451c7d8-6a5f-4d30-a24d-076da7182629'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.618478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] The token cache is being invalidate because project e833517b53454447915a423c49167866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:34.618791 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.619898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1451c7d8-6a5f-4d30-a24d-076da7182629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:34.621537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1328/2654] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:34 2022] DELETE /identity/v3/projects/e833517b53454447915a423c49167866 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:34.626569 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947826b-42f3-4950-9672-38a12d006dff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.627097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947826b-42f3-4950-9672-38a12d006dff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.627187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947826b-42f3-4950-9672-38a12d006dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.646980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4947826b-42f3-4950-9672-38a12d006dff None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:34.648178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4947826b-42f3-4950-9672-38a12d006dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:34.718824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1327/2655] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:34.726852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5db47-0a5d-4ed5-89bd-88a623c2c507 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.727173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5db47-0a5d-4ed5-89bd-88a623c2c507 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.727457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5db47-0a5d-4ed5-89bd-88a623c2c507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.748982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e4e5db47-0a5d-4ed5-89bd-88a623c2c507 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:34.750265 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e4e5db47-0a5d-4ed5-89bd-88a623c2c507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:34.758347 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1329/2656] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:34.763027 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947826b-42f3-4950-9672-38a12d006dff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:34.773930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947826b-42f3-4950-9672-38a12d006dff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:34.779028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:34.780221 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.780505 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.780779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.781699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:34.781871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:34.783530 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:34.796105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1328/2657] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:34.800284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5db47-0a5d-4ed5-89bd-88a623c2c507 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:34.809521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5db47-0a5d-4ed5-89bd-88a623c2c507 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:34.814734 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:34.815389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.815677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.815956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.816657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:34.818605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:34.846533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1330/2658] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:34 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:34.850289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:34.861532 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a5575-7523-43df-a9b7-6ad21b44bc65 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:34.942980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:34.943687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.943982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.944253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.944989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:34.946945 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:34.972502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1329/2659] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:34 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:34.976565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:34.987652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533d3be-0234-4679-910e-8bc719c304f0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:34.992309 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:34.992973 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:34.993294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:34.993571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:34.994295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:34.995927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.006585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1331/2660] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:35.010949 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.021396 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949673d-4c26-43cd-b337-aa2fa05b93e3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.026575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e78dd-efc5-4047-8829-a87b68dd6261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.027268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e78dd-efc5-4047-8829-a87b68dd6261 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.027554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e78dd-efc5-4047-8829-a87b68dd6261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.027835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e78dd-efc5-4047-8829-a87b68dd6261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.028663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e78dd-efc5-4047-8829-a87b68dd6261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.030828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e78dd-efc5-4047-8829-a87b68dd6261 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.041631 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1330/2661] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:35.046418 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.057493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab684-8d1e-4a50-9b32-cc4b88bddb69 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.062217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.063104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.063386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.063658 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] PATH_INFO: `/v3/projects/0cd88dc0b84d47e394847f12a51ed21a/users/3008a29588e34b12b91d5adffe0ac78f/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.078450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd88dc0b84d47e394847f12a51ed21a, user_id=3008a29588e34b12b91d5adffe0ac78f, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.081206 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.094984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1332/2662] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:35 2022] PUT /identity/v3/projects/0cd88dc0b84d47e394847f12a51ed21a/users/3008a29588e34b12b91d5adffe0ac78f/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:35.106432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700349-3de8-40c8-b80c-dc0e776f7618 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.106735 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700349-3de8-40c8-b80c-dc0e776f7618 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.107245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700349-3de8-40c8-b80c-dc0e776f7618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.135536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-64700349-3de8-40c8-b80c-dc0e776f7618 None None] MFA Rules not processed for user `3008a29588e34b12b91d5adffe0ac78f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:35.136997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-64700349-3de8-40c8-b80c-dc0e776f7618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:35.166703 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:35.204153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1331/2663] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:35.208689 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.215548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:35.272673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac12f7-27d2-43e6-a29d-183d67469ecd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.277298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.277954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.278240 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.278517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.279867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.281804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.306380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1333/2664] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:35 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:35.310489 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64700349-3de8-40c8-b80c-dc0e776f7618 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.321078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64700349-3de8-40c8-b80c-dc0e776f7618 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.376551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.377247 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.377534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.377816 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.378534 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.380504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.400507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1332/2665] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:35 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:35.405004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.415267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a93f-46e7-419a-80a8-6dae1d0278b0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.419816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.420479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.420764 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.421042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.421982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.424668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.436672 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1334/2666] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:35.441026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.451611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96bb30-b4e4-4371-b42b-671a3cff1ca5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.456432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.457328 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.457609 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.457879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] PATH_INFO: `/v3/projects/b26aa052188546789e91db0661dbbb9d/users/127d96244dd5494d9eaa27bfb09616fa/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.474030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26aa052188546789e91db0661dbbb9d, user_id=127d96244dd5494d9eaa27bfb09616fa, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.476660 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.490206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1333/2667] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:35 2022] PUT /identity/v3/projects/b26aa052188546789e91db0661dbbb9d/users/127d96244dd5494d9eaa27bfb09616fa/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:35.494094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.504461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908d2b0-9e70-4699-864a-0b2585beef4b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.560929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678562b-214e-43c1-a6df-ba7d29d517bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.561711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678562b-214e-43c1-a6df-ba7d29d517bc admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.562028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678562b-214e-43c1-a6df-ba7d29d517bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.562306 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678562b-214e-43c1-a6df-ba7d29d517bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.563020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678562b-214e-43c1-a6df-ba7d29d517bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.564654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678562b-214e-43c1-a6df-ba7d29d517bc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.574278 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1335/2668] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:35.578520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.588448 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424424d-a378-4fd9-abfb-ae404891c256 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.593520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.594390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.594667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.594935 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] PATH_INFO: `/v3/projects/b26aa052188546789e91db0661dbbb9d/users/127d96244dd5494d9eaa27bfb09616fa/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.596880 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26aa052188546789e91db0661dbbb9d, user_id=127d96244dd5494d9eaa27bfb09616fa, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.599494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.610160 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1334/2669] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:35 2022] PUT /identity/v3/projects/b26aa052188546789e91db0661dbbb9d/users/127d96244dd5494d9eaa27bfb09616fa/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:35.619889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02ba6-4e3e-425d-9942-dc0811b6128a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.620205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02ba6-4e3e-425d-9942-dc0811b6128a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.620499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d02ba6-4e3e-425d-9942-dc0811b6128a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.646687 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-02d02ba6-4e3e-425d-9942-dc0811b6128a None None] MFA Rules not processed for user `127d96244dd5494d9eaa27bfb09616fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:35.647887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-02d02ba6-4e3e-425d-9942-dc0811b6128a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:35.712122 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:35.716401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1336/2670] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:35.726879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:35.728663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:35.810268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c502c-eba5-4f95-8fd6-fbd0c4f5231a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:35.814216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89107fd-b2af-4c5a-b2db-9d91aae200bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:35.814859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89107fd-b2af-4c5a-b2db-9d91aae200bf service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:35.815143 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89107fd-b2af-4c5a-b2db-9d91aae200bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:35.815418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89107fd-b2af-4c5a-b2db-9d91aae200bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:35.824511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89107fd-b2af-4c5a-b2db-9d91aae200bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:35.825964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89107fd-b2af-4c5a-b2db-9d91aae200bf service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:35.843175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1335/2671] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:35.972030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d02ba6-4e3e-425d-9942-dc0811b6128a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:36.040288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d02ba6-4e3e-425d-9942-dc0811b6128a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:36.044977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:36.046169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:36.047002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:36.047002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] PATH_INFO: `/v3/users/3008a29588e34b12b91d5adffe0ac78f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:36.048779 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3008a29588e34b12b91d5adffe0ac78f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:36.051922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:36.105977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3008a29588e34b12b91d5adffe0ac78f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.124840 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3008a29588e34b12b91d5adffe0ac78f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.129555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3008a29588e34b12b91d5adffe0ac78f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.130369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3008a29588e34b12b91d5adffe0ac78f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.151587 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3008a29588e34b12b91d5adffe0ac78f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.170615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1337/2672] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:35 2022] DELETE /identity/v3/users/3008a29588e34b12b91d5adffe0ac78f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:36.177335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:36.177335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:36.177335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:36.194608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:36.199301 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:36.201516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:36.296174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1336/2673] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:36.303507 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:36.304902 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:36.380778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af7669-66cb-4ba3-88f7-371c76dd0da9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:36.384403 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:36.385057 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:36.385457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:36.385650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:36.398229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:36.399746 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:36.410281 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1338/2674] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:36.705503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:36.707132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:36.737179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5196d6-f442-4d20-96a2-bbc5d0f95b9e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:36.742399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:36.743389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:36.743679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:36.743957 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] PATH_INFO: `/v3/projects/0cd88dc0b84d47e394847f12a51ed21a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:36.745184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd88dc0b84d47e394847f12a51ed21a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:36.747230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:36.801207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd88dc0b84d47e394847f12a51ed21a', 'request_id': 'req-83c83a55-05ca-4d8c-a766-32bb46474ca6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.809893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd88dc0b84d47e394847f12a51ed21a', 'request_id': 'req-83c83a55-05ca-4d8c-a766-32bb46474ca6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.817644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd88dc0b84d47e394847f12a51ed21a', 'request_id': 'req-83c83a55-05ca-4d8c-a766-32bb46474ca6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.832365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] The token cache is being invalidate because project 0cd88dc0b84d47e394847f12a51ed21a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:36.832701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.833377 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.834933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1337/2675] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:36 2022] DELETE /identity/v3/projects/0cd88dc0b84d47e394847f12a51ed21a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:36.838803 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:36.848343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7ac1-7017-46c7-807f-eb0129978826 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:36.867151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:36.908024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:36.908942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:36.909255 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:36.909534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] PATH_INFO: `/v3/users/127d96244dd5494d9eaa27bfb09616fa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:36.910726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127d96244dd5494d9eaa27bfb09616fa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:36.912648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:36.958270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127d96244dd5494d9eaa27bfb09616fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.971754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127d96244dd5494d9eaa27bfb09616fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.972527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127d96244dd5494d9eaa27bfb09616fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.973138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127d96244dd5494d9eaa27bfb09616fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.981583 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a942f5a-7d58-4b60-8b88-f7eb8ea83849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127d96244dd5494d9eaa27bfb09616fa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:36.996760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1339/2676] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:36 2022] DELETE /identity/v3/users/127d96244dd5494d9eaa27bfb09616fa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:37.270809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:37.272354 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:37.287351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c83a55-05ca-4d8c-a766-32bb46474ca6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:37.371501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:37.372495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.373321 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.373602 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] PATH_INFO: `/v3/projects/b26aa052188546789e91db0661dbbb9d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.374804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b26aa052188546789e91db0661dbbb9d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:37.376713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:37.412230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:37.426765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b26aa052188546789e91db0661dbbb9d', 'request_id': 'req-551d469d-3fd5-4d08-8536-1346e5d58bbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:37.435023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b26aa052188546789e91db0661dbbb9d', 'request_id': 'req-551d469d-3fd5-4d08-8536-1346e5d58bbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:37.443875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b26aa052188546789e91db0661dbbb9d', 'request_id': 'req-551d469d-3fd5-4d08-8536-1346e5d58bbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:37.457762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] The token cache is being invalidate because project b26aa052188546789e91db0661dbbb9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:37.458280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:37.459046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551d469d-3fd5-4d08-8536-1346e5d58bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:37.460628 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1338/2677] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:37 2022] DELETE /identity/v3/projects/b26aa052188546789e91db0661dbbb9d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:37.466514 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9437fc9-e2cc-49f6-9d4b-92f5458a321f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.467055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9437fc9-e2cc-49f6-9d4b-92f5458a321f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.467055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9437fc9-e2cc-49f6-9d4b-92f5458a321f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.485370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d9437fc9-e2cc-49f6-9d4b-92f5458a321f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:37.486600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d9437fc9-e2cc-49f6-9d4b-92f5458a321f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:37.548886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1340/2678] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:37.557918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84890861-22c5-4def-98b4-a1de9d603956 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.558216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84890861-22c5-4def-98b4-a1de9d603956 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.558501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84890861-22c5-4def-98b4-a1de9d603956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.577510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-84890861-22c5-4def-98b4-a1de9d603956 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:37.578713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84890861-22c5-4def-98b4-a1de9d603956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:37.586567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1339/2679] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:37.590681 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9437fc9-e2cc-49f6-9d4b-92f5458a321f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:37.600127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9437fc9-e2cc-49f6-9d4b-92f5458a321f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:37.605007 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:37.605708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.606108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.606398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.607680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:37.608020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:37.609656 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:37.621445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1341/2680] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:37.626175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84890861-22c5-4def-98b4-a1de9d603956 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:37.635284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84890861-22c5-4def-98b4-a1de9d603956 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:37.640172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:37.640819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.641106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.641415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.642115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:37.644022 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:37.668830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1340/2681] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:37 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:37.673457 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:37.683393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe540dd-442a-4af7-9baa-627567e81f05 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:37.709407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:37.742353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:37.743024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.743315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.743591 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.744276 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:37.746237 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:37.767334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1342/2682] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:37 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:37.771343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:37.781593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4ebd4-d3f3-4270-ae2a-82ba6584efa4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:37.786202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:37.786860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.787146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.787420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.788155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:37.789813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:37.801614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1341/2683] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:37.805424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:37.816235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd94e9-1ad6-4b22-b20d-03b2a3d6b4f9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:37.820813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a73246-c9ce-49ab-89e2-c45e3c426029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:37.821525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73246-c9ce-49ab-89e2-c45e3c426029 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.821807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73246-c9ce-49ab-89e2-c45e3c426029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.822083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73246-c9ce-49ab-89e2-c45e3c426029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.822796 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a73246-c9ce-49ab-89e2-c45e3c426029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:37.824413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a73246-c9ce-49ab-89e2-c45e3c426029 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:37.834602 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1343/2684] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:37.838544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:37.848821 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9367c-e822-4204-929b-5f5dfe169c28 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:37.853393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:37.854516 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.854802 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.855074 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] PATH_INFO: `/v3/projects/eb46d0c1b2c14024822577d539627af4/users/56c79535f68f4b8bb60156291301b8ba/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.870915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb46d0c1b2c14024822577d539627af4, user_id=56c79535f68f4b8bb60156291301b8ba, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:37.873605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:37.887034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1342/2685] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:37 2022] PUT /identity/v3/projects/eb46d0c1b2c14024822577d539627af4/users/56c79535f68f4b8bb60156291301b8ba/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:37.898112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c5d00-f155-40a7-82fc-2594ebe459c3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:37.898415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c5d00-f155-40a7-82fc-2594ebe459c3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:37.898719 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c5d00-f155-40a7-82fc-2594ebe459c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:37.931811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-196c5d00-f155-40a7-82fc-2594ebe459c3 None None] MFA Rules not processed for user `56c79535f68f4b8bb60156291301b8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:37.933014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-196c5d00-f155-40a7-82fc-2594ebe459c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.000368 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.000554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1344/2686] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:38.002148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.004579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c5d00-f155-40a7-82fc-2594ebe459c3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.067627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c5d00-f155-40a7-82fc-2594ebe459c3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.067933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7adb-2f99-4d90-8155-b642a7f3cfbf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.072272 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.072938 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.073129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.073273 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.073538 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.074037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.074229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.074329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.074581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] PATH_INFO: `/v3/users/dc6be2a4a55147dcb864e44b82f24ed8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.075785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6be2a4a55147dcb864e44b82f24ed8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.076149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.077801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.103313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1345/2687] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:38 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:38.108000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.127113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6be2a4a55147dcb864e44b82f24ed8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.134778 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6be2a4a55147dcb864e44b82f24ed8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.134983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:38.140539 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:38.150285 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6be2a4a55147dcb864e44b82f24ed8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.150993 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6be2a4a55147dcb864e44b82f24ed8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.151794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-686aaa09-7bd0-4800-967f-d362d5137ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6be2a4a55147dcb864e44b82f24ed8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.165202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1343/2688] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:37 2022] DELETE /identity/v3/users/dc6be2a4a55147dcb864e44b82f24ed8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:38.169578 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.170363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.170647 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.175799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b429d-f986-48f7-b8c9-9dfee6b2081e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.189500 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:38.190711 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.235327 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.236005 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.236292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.236568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.237305 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.239259 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.254172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1344/2689] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:38.263044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1346/2690] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:38 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:38.264205 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.264358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.265637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.265945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.277300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762f15d-f39c-4e64-8ba7-d8f904ee45ff admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.282223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.282884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.283179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.283462 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.284191 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.285837 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.296370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1347/2691] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:38.300186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.309920 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb2afe-66be-4be6-9467-b7c7fea9ab84 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.315289 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.316030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.316319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.316594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.317337 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.319037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.325044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b65d10-cf87-4192-8fe9-51eccbac7e83 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.328939 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4e8dd-e232-4e8f-a557-37cb8a3b80ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.330647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1348/2692] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:38.330779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4e8dd-e232-4e8f-a557-37cb8a3b80ff service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.330779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4e8dd-e232-4e8f-a557-37cb8a3b80ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.330779 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4e8dd-e232-4e8f-a557-37cb8a3b80ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.333907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.342120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4e8dd-e232-4e8f-a557-37cb8a3b80ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.343317 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3dabb-00f8-4989-ac87-24694d6b1f6d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.343769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4e8dd-e232-4e8f-a557-37cb8a3b80ff service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.347951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.348804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.349094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.349401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] PATH_INFO: `/v3/projects/4719751a1a4749d9b18bb56ff8e4d1ba/users/13f7e1717b2e4ab3bc199b05973cd4a9/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.354731 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1345/2693] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:38.364577 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4719751a1a4749d9b18bb56ff8e4d1ba, user_id=13f7e1717b2e4ab3bc199b05973cd4a9, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.367259 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.382692 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1349/2694] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:38 2022] PUT /identity/v3/projects/4719751a1a4749d9b18bb56ff8e4d1ba/users/13f7e1717b2e4ab3bc199b05973cd4a9/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:38.393264 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404a616-83ae-4760-a488-ad9b4d1d948c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.394106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404a616-83ae-4760-a488-ad9b4d1d948c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.394398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404a616-83ae-4760-a488-ad9b4d1d948c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.424946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9404a616-83ae-4760-a488-ad9b4d1d948c None None] MFA Rules not processed for user `13f7e1717b2e4ab3bc199b05973cd4a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:38.426363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9404a616-83ae-4760-a488-ad9b4d1d948c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.496415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1346/2695] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:38.501594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.586450 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0f553-670a-42ce-bbf6-55096240b57f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.592187 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.592566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.592949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.593336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.594118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.596180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.617792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:38.626619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1350/2696] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:38 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:38.630608 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404a616-83ae-4760-a488-ad9b4d1d948c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.641801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404a616-83ae-4760-a488-ad9b4d1d948c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.646566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:38.706783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.707918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.708456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.709139 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.710279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.712549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.725478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.736921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1347/2697] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:38 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:38.737964 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.740819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.751099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021b56-b5c3-40d0-abde-1bf2bff74226 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.751708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b246474-8526-412a-910c-f95142ba6ac5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.756130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.756992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.757280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.757637 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.757915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.758271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.759312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.759751 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.759954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.760428 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] PATH_INFO: `/v3/projects/bdf99ab130464dd0a4babe0da87c0fde` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.762249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf99ab130464dd0a4babe0da87c0fde)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.764373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.771073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1348/2698] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:38.775042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.785260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc5fb-54c1-4fee-86e1-349e2ce17585 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.790083 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.790977 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.791263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.791543 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] PATH_INFO: `/v3/projects/0c65ea7481e34660b6345056f90e0334/users/9af220f3343c4f5f8f78f64ed44694b7/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.809265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c65ea7481e34660b6345056f90e0334, user_id=9af220f3343c4f5f8f78f64ed44694b7, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.811997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.816685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf99ab130464dd0a4babe0da87c0fde', 'request_id': 'req-174d5110-c3a0-4da2-94fc-c9a025ddd003'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.823865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf99ab130464dd0a4babe0da87c0fde', 'request_id': 'req-174d5110-c3a0-4da2-94fc-c9a025ddd003'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.826157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1349/2699] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:38 2022] PUT /identity/v3/projects/0c65ea7481e34660b6345056f90e0334/users/9af220f3343c4f5f8f78f64ed44694b7/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:38.829810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.830104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf99ab130464dd0a4babe0da87c0fde', 'request_id': 'req-174d5110-c3a0-4da2-94fc-c9a025ddd003'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.843647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] The token cache is being invalidate because project bdf99ab130464dd0a4babe0da87c0fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:38.844178 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.844825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.846650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1351/2700] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:38 2022] DELETE /identity/v3/projects/bdf99ab130464dd0a4babe0da87c0fde => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:38.850209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.851572 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.894994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38a94a-1d6b-45d6-a694-bd279fac6e93 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.899749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.900417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.900704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.900987 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.901751 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.903360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.917384 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1350/2701] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:38.917562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d5110-c3a0-4da2-94fc-c9a025ddd003 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.918476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:38.920786 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.923017 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.923893 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.924176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.924452 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] PATH_INFO: `/v3/users/f4d7337d63194363aefefc158ee1ba0a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.925668 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d7337d63194363aefefc158ee1ba0a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.927598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.933305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a3823-b497-4586-b2b4-7dbb319c6d15 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:38.937885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f446a9-eff4-46c1-b5bd-ed74f0e9e675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:38.938757 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f446a9-eff4-46c1-b5bd-ed74f0e9e675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.939155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f446a9-eff4-46c1-b5bd-ed74f0e9e675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.939441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f446a9-eff4-46c1-b5bd-ed74f0e9e675 admin admin] PATH_INFO: `/v3/projects/0c65ea7481e34660b6345056f90e0334/users/9af220f3343c4f5f8f78f64ed44694b7/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.941512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f446a9-eff4-46c1-b5bd-ed74f0e9e675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c65ea7481e34660b6345056f90e0334, user_id=9af220f3343c4f5f8f78f64ed44694b7, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:38.944118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f446a9-eff4-46c1-b5bd-ed74f0e9e675 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:38.955404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1351/2702] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:38 2022] PUT /identity/v3/projects/0c65ea7481e34660b6345056f90e0334/users/9af220f3343c4f5f8f78f64ed44694b7/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:38.965452 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:38.965746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:38.966029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:38.975562 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d7337d63194363aefefc158ee1ba0a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.988253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d7337d63194363aefefc158ee1ba0a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.988975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d7337d63194363aefefc158ee1ba0a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.989639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d7337d63194363aefefc158ee1ba0a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:38.992663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] MFA Rules not processed for user `9af220f3343c4f5f8f78f64ed44694b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:38.994045 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:38.997712 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d7337d63194363aefefc158ee1ba0a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:39.006313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:39.012477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1352/2703] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:38 2022] DELETE /identity/v3/users/f4d7337d63194363aefefc158ee1ba0a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:39.030611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:39.071071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1352/2704] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:39.082568 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.084046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:39.146497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a57f-22cb-434e-afb0-f7cbd066e38c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.150488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88045b7c-bf9f-40d7-9eec-0cb11d2a87fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.151077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88045b7c-bf9f-40d7-9eec-0cb11d2a87fd service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.151367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88045b7c-bf9f-40d7-9eec-0cb11d2a87fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.151648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88045b7c-bf9f-40d7-9eec-0cb11d2a87fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.154520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-88045b7c-bf9f-40d7-9eec-0cb11d2a87fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:39.226048 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88045b7c-bf9f-40d7-9eec-0cb11d2a87fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.227505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88045b7c-bf9f-40d7-9eec-0cb11d2a87fd service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.236883 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1353/2705] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:39.294556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.297057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:39.376451 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5be84-1b66-4791-806b-6b8c8c77b064 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.436938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.437834 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.438116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.438390 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] PATH_INFO: `/v3/projects/98a045e9fa814cdbaa5d1dcb41f4b0de` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.439558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a045e9fa814cdbaa5d1dcb41f4b0de)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.441503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.494984 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a045e9fa814cdbaa5d1dcb41f4b0de', 'request_id': 'req-98be4fa6-8f98-41d2-8cca-f41eab97455d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:39.503407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a045e9fa814cdbaa5d1dcb41f4b0de', 'request_id': 'req-98be4fa6-8f98-41d2-8cca-f41eab97455d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:39.510759 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a045e9fa814cdbaa5d1dcb41f4b0de', 'request_id': 'req-98be4fa6-8f98-41d2-8cca-f41eab97455d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:39.523667 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] The token cache is being invalidate because project 98a045e9fa814cdbaa5d1dcb41f4b0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:39.523959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:39.524617 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98be4fa6-8f98-41d2-8cca-f41eab97455d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:39.526756 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1353/2706] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:39 2022] DELETE /identity/v3/projects/98a045e9fa814cdbaa5d1dcb41f4b0de => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:39.532400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e340ff3-941e-4ece-bdff-fa0493c58d43 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.532935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e340ff3-941e-4ece-bdff-fa0493c58d43 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.533047 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e340ff3-941e-4ece-bdff-fa0493c58d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.551459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1e340ff3-941e-4ece-bdff-fa0493c58d43 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:39.552839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1e340ff3-941e-4ece-bdff-fa0493c58d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:39.617356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1354/2707] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:39.625085 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013defb-86ff-40cf-9753-2b6fcbffa5cf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.625422 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013defb-86ff-40cf-9753-2b6fcbffa5cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.625705 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013defb-86ff-40cf-9753-2b6fcbffa5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.644982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f013defb-86ff-40cf-9753-2b6fcbffa5cf None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:39.646174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f013defb-86ff-40cf-9753-2b6fcbffa5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:39.654732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1354/2708] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:39.659019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e340ff3-941e-4ece-bdff-fa0493c58d43 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.668560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e340ff3-941e-4ece-bdff-fa0493c58d43 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.679064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.679738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.680024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.680301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.681074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:39.681422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.683102 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.694966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1355/2709] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:39.699393 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013defb-86ff-40cf-9753-2b6fcbffa5cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.704402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:39.710648 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013defb-86ff-40cf-9753-2b6fcbffa5cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.716151 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.717437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.717718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.717988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.718674 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.720563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.745712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1355/2710] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:39 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:39.749921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.759977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2647c5-6d58-45a8-b6db-18244c5badcb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.801800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:39.818590 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.819270 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.819558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.819838 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.820541 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.822499 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.845853 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1356/2711] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:39 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:39.849975 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.860250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6bcb-3a00-4984-94f4-79a9e7d2d056 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.865133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.865836 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.866118 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.866389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.867107 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.868685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.880253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1356/2712] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:39.884235 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.895824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df39700-16a5-4726-968d-e923ef811ff4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.901505 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e11b1-7dd1-4a62-93f9-efb37ab07b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.902250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e11b1-7dd1-4a62-93f9-efb37ab07b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.902540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e11b1-7dd1-4a62-93f9-efb37ab07b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.902816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e11b1-7dd1-4a62-93f9-efb37ab07b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.903547 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e11b1-7dd1-4a62-93f9-efb37ab07b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.905774 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e11b1-7dd1-4a62-93f9-efb37ab07b8b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.916127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1357/2713] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:39.920554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:39.930592 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cf4b-d024-4c59-b78c-4ecc520c0f6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:39.935232 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:39.936106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.936389 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.936664 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] PATH_INFO: `/v3/projects/90e9c8cc32ae4b1ba76712bbd1768eca/users/48e6317bad1842e180ffae0c0000bfa4/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:39.951266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e9c8cc32ae4b1ba76712bbd1768eca, user_id=48e6317bad1842e180ffae0c0000bfa4, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:39.953930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:39.967059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1357/2714] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:39 2022] PUT /identity/v3/projects/90e9c8cc32ae4b1ba76712bbd1768eca/users/48e6317bad1842e180ffae0c0000bfa4/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:39.978548 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f73ce-755f-4e4e-bf94-b3dd014683f1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:39.978844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f73ce-755f-4e4e-bf94-b3dd014683f1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:39.979129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f73ce-755f-4e4e-bf94-b3dd014683f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.005288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7b3f73ce-755f-4e4e-bf94-b3dd014683f1 None None] MFA Rules not processed for user `48e6317bad1842e180ffae0c0000bfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:40.006494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7b3f73ce-755f-4e4e-bf94-b3dd014683f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.069940 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1358/2715] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:40.082125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:40.083807 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.140499 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36991899-aaf8-4508-b973-ae186ca0e9ff admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:40.144076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f67f6f-6f7c-475e-ae80-adea737da8ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:40.144712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67f6f-6f7c-475e-ae80-adea737da8ef service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:40.144991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67f6f-6f7c-475e-ae80-adea737da8ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:40.145298 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67f6f-6f7c-475e-ae80-adea737da8ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.153606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f67f6f-6f7c-475e-ae80-adea737da8ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:40.155158 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f67f6f-6f7c-475e-ae80-adea737da8ef service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:40.164038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1358/2716] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:40.182921 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f73ce-755f-4e4e-bf94-b3dd014683f1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:40.244140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f73ce-755f-4e4e-bf94-b3dd014683f1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:40.249201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:40.250070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:40.250353 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:40.250652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] PATH_INFO: `/v3/users/48e6317bad1842e180ffae0c0000bfa4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.251826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e6317bad1842e180ffae0c0000bfa4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:40.253784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:40.299975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e6317bad1842e180ffae0c0000bfa4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.304362 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:40.314083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e6317bad1842e180ffae0c0000bfa4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.314842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e6317bad1842e180ffae0c0000bfa4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.315647 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e6317bad1842e180ffae0c0000bfa4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.322735 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e6317bad1842e180ffae0c0000bfa4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.336091 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1359/2717] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:40 2022] DELETE /identity/v3/users/48e6317bad1842e180ffae0c0000bfa4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:40.340698 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:40.340996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:40.341373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.358372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:40.359570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.417839 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1359/2718] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:40.430271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:40.431730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.489599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a817fa-55d9-468b-97fd-96ec1f915c5e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:40.493186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:40.493821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:40.494099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:40.494373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.504138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:40.505652 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:40.515208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1360/2719] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:40.763278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:40.764867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.797364 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:40.798907 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.812945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a3c7b-732c-4035-9954-2cdf82fe0ec2 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:40.819053 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:40.819850 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:40.820292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:40.820684 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] PATH_INFO: `/v3/projects/90e9c8cc32ae4b1ba76712bbd1768eca` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.821898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e9c8cc32ae4b1ba76712bbd1768eca)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:40.823809 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:40.829217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:40.834028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e4124-ab37-4b24-a0b0-1f694047c2c3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:40.839196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff132ee6-c3e1-471d-ad61-fb72499245a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:40.840121 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff132ee6-c3e1-471d-ad61-fb72499245a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:40.840597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff132ee6-c3e1-471d-ad61-fb72499245a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:40.840884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff132ee6-c3e1-471d-ad61-fb72499245a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.843296 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ff132ee6-c3e1-471d-ad61-fb72499245a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.877502 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e9c8cc32ae4b1ba76712bbd1768eca', 'request_id': 'req-02b55e55-9753-4944-bf10-5555019d4b93'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.883713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e9c8cc32ae4b1ba76712bbd1768eca', 'request_id': 'req-02b55e55-9753-4944-bf10-5555019d4b93'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.891062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e9c8cc32ae4b1ba76712bbd1768eca', 'request_id': 'req-02b55e55-9753-4944-bf10-5555019d4b93'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.903942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] The token cache is being invalidate because project 90e9c8cc32ae4b1ba76712bbd1768eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:40.904234 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.904873 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02b55e55-9753-4944-bf10-5555019d4b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:40.906472 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1361/2720] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:40 2022] DELETE /identity/v3/projects/90e9c8cc32ae4b1ba76712bbd1768eca => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:40.907267 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff132ee6-c3e1-471d-ad61-fb72499245a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:40.908806 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff132ee6-c3e1-471d-ad61-fb72499245a5 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:40.911073 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab18e20-10a5-4101-8773-750e7dd6cdca None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:40.912707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab18e20-10a5-4101-8773-750e7dd6cdca None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:40.912707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab18e20-10a5-4101-8773-750e7dd6cdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:40.918781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1360/2721] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:40.931909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-eab18e20-10a5-4101-8773-750e7dd6cdca None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:40.933116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eab18e20-10a5-4101-8773-750e7dd6cdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:40.990953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:41.014195 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1362/2722] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:41.022317 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c5c3-3125-4b21-bd12-a5dd92f57a7d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.022625 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c5c3-3125-4b21-bd12-a5dd92f57a7d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.022902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c5c3-3125-4b21-bd12-a5dd92f57a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.045075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e978c5c3-3125-4b21-bd12-a5dd92f57a7d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:41.046743 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e978c5c3-3125-4b21-bd12-a5dd92f57a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:41.057715 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1361/2723] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:41.062365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab18e20-10a5-4101-8773-750e7dd6cdca None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.078358 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab18e20-10a5-4101-8773-750e7dd6cdca None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.084323 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.085249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.085608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.085951 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.086821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:41.087194 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.088926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.105201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1363/2724] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:41.107960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978c5c3-3125-4b21-bd12-a5dd92f57a7d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.119570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978c5c3-3125-4b21-bd12-a5dd92f57a7d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.125385 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.126535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.127076 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.127619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.128864 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.132507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.163445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1362/2725] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:41 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:41.167992 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.179055 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a0c0a-12a3-419a-b5c8-64fe50e224f6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.240503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.241205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.241497 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.241783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.242486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.244415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.273357 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.275302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1364/2726] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:41 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:41.284859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a09038-a5b9-486e-b4ab-cbac59cbbb25 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.290394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.291268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.291575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.291878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.292781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.294440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.305184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1363/2727] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:08:41.309945 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.321064 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338741a-9c53-4067-9f78-cc42c717636d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.326341 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9f5e5-047e-4fa2-84ba-1f0b25219da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.327041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f5e5-047e-4fa2-84ba-1f0b25219da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.327453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f5e5-047e-4fa2-84ba-1f0b25219da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.327742 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f5e5-047e-4fa2-84ba-1f0b25219da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.328476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9f5e5-047e-4fa2-84ba-1f0b25219da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.330170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9f5e5-047e-4fa2-84ba-1f0b25219da0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.342220 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1365/2728] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:41.346524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.356606 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbf29b-6098-4182-b524-ed0ed416e4e5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.362335 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.363228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.363517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.363797 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] PATH_INFO: `/v3/projects/787413a07a3c406b80e3bc82f03f092d/users/5c30a10b60d3487294a9610720a72992/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.379923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=787413a07a3c406b80e3bc82f03f092d, user_id=5c30a10b60d3487294a9610720a72992, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.383025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.398570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1364/2729] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:41 2022] PUT /identity/v3/projects/787413a07a3c406b80e3bc82f03f092d/users/5c30a10b60d3487294a9610720a72992/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:41.409280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.409579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.409863 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.436965 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] MFA Rules not processed for user `5c30a10b60d3487294a9610720a72992`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:41.438211 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:41.507361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1366/2730] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:41.520398 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.522261 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:41.526404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:41.527768 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:41.587226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97aa7e-9651-4363-9ad8-e1083138f29b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.589030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52dabba-247a-4477-a55d-fcb1d28502fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:41.591346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc11d1-236c-4fa2-b401-3cb40145fdbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.592003 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc11d1-236c-4fa2-b401-3cb40145fdbb service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.592316 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc11d1-236c-4fa2-b401-3cb40145fdbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.592594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc11d1-236c-4fa2-b401-3cb40145fdbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.593322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:41.593969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:41.594248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:41.594417 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-88fc11d1-236c-4fa2-b401-3cb40145fdbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:41.594529 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:41.599481 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:41.604125 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.605706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.615531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1365/2731] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:41.671210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc11d1-236c-4fa2-b401-3cb40145fdbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:41.673471 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc11d1-236c-4fa2-b401-3cb40145fdbb service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:41.683980 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1367/2732] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:08:48.731172 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:48.733023 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:48.782782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:48.806270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb669e-2497-40b5-90f1-bd8336eeef80 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:48.810837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:48.811776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:48.812053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:48.812334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] PATH_INFO: `/v3/users/56c79535f68f4b8bb60156291301b8ba` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:48.813567 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c79535f68f4b8bb60156291301b8ba)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:48.815490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:48.864846 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c79535f68f4b8bb60156291301b8ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:48.872325 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c79535f68f4b8bb60156291301b8ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:48.886327 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c79535f68f4b8bb60156291301b8ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:48.887005 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c79535f68f4b8bb60156291301b8ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:48.887820 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c79535f68f4b8bb60156291301b8ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:48.902153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1366/2733] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:48 2022] DELETE /identity/v3/users/56c79535f68f4b8bb60156291301b8ba => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:48.907449 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:48.907906 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:48.908016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:48.925635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:48.926927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:48.986231 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1368/2734] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:48.995761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:48.997229 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:49.056332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ee32-8a0f-40ad-b998-b6ae21941f93 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:49.059942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:49.060579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:49.060885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:49.061177 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:49.070914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:49.072332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:49.082918 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1367/2735] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:49.365133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:49.366612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:49.384216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fafaf-4373-412b-adac-bcb72fda3d82 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:49.390470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:49.390826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:49.391114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:49.391311 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] PATH_INFO: `/v3/projects/eb46d0c1b2c14024822577d539627af4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:49.392509 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb46d0c1b2c14024822577d539627af4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:49.394423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:49.441944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb46d0c1b2c14024822577d539627af4', 'request_id': 'req-98364943-8a43-4623-83e7-af9bae575932'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.447991 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb46d0c1b2c14024822577d539627af4', 'request_id': 'req-98364943-8a43-4623-83e7-af9bae575932'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.453990 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb46d0c1b2c14024822577d539627af4', 'request_id': 'req-98364943-8a43-4623-83e7-af9bae575932'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.467295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] The token cache is being invalidate because project eb46d0c1b2c14024822577d539627af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:49.467625 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.468394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.470026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1369/2736] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:49 2022] DELETE /identity/v3/projects/eb46d0c1b2c14024822577d539627af4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:49.474169 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:49.484084 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6948e-8afc-4f6d-bb85-60ffe7418741 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:49.543460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:49.544245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:49.544525 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:49.544803 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] PATH_INFO: `/v3/users/13f7e1717b2e4ab3bc199b05973cd4a9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:49.546495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f7e1717b2e4ab3bc199b05973cd4a9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:49.548406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:49.552823 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:49.595248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f7e1717b2e4ab3bc199b05973cd4a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.602343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f7e1717b2e4ab3bc199b05973cd4a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.614630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f7e1717b2e4ab3bc199b05973cd4a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.615265 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f7e1717b2e4ab3bc199b05973cd4a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.615861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f7e1717b2e4ab3bc199b05973cd4a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:49.628456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1368/2737] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:49 2022] DELETE /identity/v3/users/13f7e1717b2e4ab3bc199b05973cd4a9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:49.906296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:49.907754 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:49.922475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98364943-8a43-4623-83e7-af9bae575932 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:49.947101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:49.997365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:49.998365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:49.998645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:49.998915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] PATH_INFO: `/v3/projects/4719751a1a4749d9b18bb56ff8e4d1ba` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:50.000184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4719751a1a4749d9b18bb56ff8e4d1ba)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:50.002140 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:50.070748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4719751a1a4749d9b18bb56ff8e4d1ba', 'request_id': 'req-3be0e71e-810a-4502-8e3c-802dcae6c04e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.077886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4719751a1a4749d9b18bb56ff8e4d1ba', 'request_id': 'req-3be0e71e-810a-4502-8e3c-802dcae6c04e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.083617 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4719751a1a4749d9b18bb56ff8e4d1ba', 'request_id': 'req-3be0e71e-810a-4502-8e3c-802dcae6c04e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.096733 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] The token cache is being invalidate because project 4719751a1a4749d9b18bb56ff8e4d1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:50.097026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.097694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.099264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1370/2738] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:49 2022] DELETE /identity/v3/projects/4719751a1a4749d9b18bb56ff8e4d1ba => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:50.103216 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:50.104620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:50.118879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd140e-8999-4f74-a09e-d480eec98596 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:50.175010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:50.175885 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:50.176165 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:50.176438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] PATH_INFO: `/v3/users/9af220f3343c4f5f8f78f64ed44694b7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:50.177655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af220f3343c4f5f8f78f64ed44694b7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:50.179560 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:50.226980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af220f3343c4f5f8f78f64ed44694b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.234464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af220f3343c4f5f8f78f64ed44694b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.246809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af220f3343c4f5f8f78f64ed44694b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.247446 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af220f3343c4f5f8f78f64ed44694b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.248051 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af220f3343c4f5f8f78f64ed44694b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.261714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1369/2739] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:50 2022] DELETE /identity/v3/users/9af220f3343c4f5f8f78f64ed44694b7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:50.564394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:50.565966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:50.579926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0e71e-810a-4502-8e3c-802dcae6c04e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:50.636930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:50.637855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:50.638132 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:50.638405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] PATH_INFO: `/v3/projects/0c65ea7481e34660b6345056f90e0334` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:50.639566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c65ea7481e34660b6345056f90e0334)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:50.643454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:50.690024 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c65ea7481e34660b6345056f90e0334', 'request_id': 'req-a377771f-6225-49c1-8e40-f49e69d3056a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.696020 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c65ea7481e34660b6345056f90e0334', 'request_id': 'req-a377771f-6225-49c1-8e40-f49e69d3056a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.697975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:50.702180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c65ea7481e34660b6345056f90e0334', 'request_id': 'req-a377771f-6225-49c1-8e40-f49e69d3056a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.714898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] The token cache is being invalidate because project 0c65ea7481e34660b6345056f90e0334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:50.715269 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.715952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a377771f-6225-49c1-8e40-f49e69d3056a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:50.717663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1371/2740] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:50 2022] DELETE /identity/v3/projects/0c65ea7481e34660b6345056f90e0334 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:51.105055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:51.106502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:51.115237 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:51.121604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e8d5-4b0a-49f5-a9fb-b1f0c0e0315f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:51.177685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:51.178566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:51.178852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:51.179130 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] PATH_INFO: `/v3/users/5c30a10b60d3487294a9610720a72992` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:51.180333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c30a10b60d3487294a9610720a72992)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:51.182290 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:51.228322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c30a10b60d3487294a9610720a72992'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.236153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c30a10b60d3487294a9610720a72992'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.247985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c30a10b60d3487294a9610720a72992'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.248629 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c30a10b60d3487294a9610720a72992'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.249871 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c30a10b60d3487294a9610720a72992'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.262284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1370/2741] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:08:51 2022] DELETE /identity/v3/users/5c30a10b60d3487294a9610720a72992 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:51.266871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:51.267355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:51.267628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:51.285947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:51.287148 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:51.349804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1372/2742] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:51.359225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:51.361223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:51.420102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622cf4-ffff-4cdb-89dd-b86fd2eab54a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:51.426184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ed1d5-5826-461a-bcd8-f44e8564c044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:51.426867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ed1d5-5826-461a-bcd8-f44e8564c044 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:51.427148 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ed1d5-5826-461a-bcd8-f44e8564c044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:51.427424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ed1d5-5826-461a-bcd8-f44e8564c044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:51.436772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ed1d5-5826-461a-bcd8-f44e8564c044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:51.438232 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ed1d5-5826-461a-bcd8-f44e8564c044 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:51.448699 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1371/2743] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:51.804732 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:51.806189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:51.824683 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b9276-eb59-4b77-b5d3-68ef1d3ede3f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:51.831199 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:51.832062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:51.832350 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:51.832621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] PATH_INFO: `/v3/projects/787413a07a3c406b80e3bc82f03f092d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:51.834188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787413a07a3c406b80e3bc82f03f092d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:51.836600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:51.882456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787413a07a3c406b80e3bc82f03f092d', 'request_id': 'req-ae13e388-6c74-40e5-93ed-88e550de7093'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.888811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787413a07a3c406b80e3bc82f03f092d', 'request_id': 'req-ae13e388-6c74-40e5-93ed-88e550de7093'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.894800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787413a07a3c406b80e3bc82f03f092d', 'request_id': 'req-ae13e388-6c74-40e5-93ed-88e550de7093'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.907380 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] The token cache is being invalidate because project 787413a07a3c406b80e3bc82f03f092d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:08:51.907666 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.908286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae13e388-6c74-40e5-93ed-88e550de7093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:08:51.909845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1373/2744] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:08:51 2022] DELETE /identity/v3/projects/787413a07a3c406b80e3bc82f03f092d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:08:51.914923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a51f8-f895-4f5a-8136-559c680b2824 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:51.915215 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a51f8-f895-4f5a-8136-559c680b2824 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:51.915495 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a51f8-f895-4f5a-8136-559c680b2824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:51.932571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f16a51f8-f895-4f5a-8136-559c680b2824 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:51.933788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f16a51f8-f895-4f5a-8136-559c680b2824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:51.954387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:51.995641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1372/2745] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:52.004389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6fb8e-a917-43cd-8e70-7a3c9142074d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.004716 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6fb8e-a917-43cd-8e70-7a3c9142074d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.005000 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6fb8e-a917-43cd-8e70-7a3c9142074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.023913 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-11c6fb8e-a917-43cd-8e70-7a3c9142074d None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:52.025108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-11c6fb8e-a917-43cd-8e70-7a3c9142074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:52.033699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1374/2746] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:08:52.037976 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a51f8-f895-4f5a-8136-559c680b2824 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:52.047574 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a51f8-f895-4f5a-8136-559c680b2824 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:52.051946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73881444-7257-4322-a0d8-733818072581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:52.052593 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73881444-7257-4322-a0d8-733818072581 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.052878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73881444-7257-4322-a0d8-733818072581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.053175 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73881444-7257-4322-a0d8-733818072581 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.053896 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73881444-7257-4322-a0d8-733818072581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:08:52.054199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73881444-7257-4322-a0d8-733818072581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:52.055781 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73881444-7257-4322-a0d8-733818072581 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:52.068696 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1373/2747] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:08:52.072769 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6fb8e-a917-43cd-8e70-7a3c9142074d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:52.083389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6fb8e-a917-43cd-8e70-7a3c9142074d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:52.092134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:52.092804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.093090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.093400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.094104 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:52.096043 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:52.126576 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1375/2748] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:08:52 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:08:52.127745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73881444-7257-4322-a0d8-733818072581 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:52.140652 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73881444-7257-4322-a0d8-733818072581 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:52.204112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:52.204780 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.205058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.205359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.206073 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:52.207991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:52.227442 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1374/2749] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:08:52 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:08:52.231701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:52.241694 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275981-d6f9-4279-bcdf-4979f4466fe6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:52.246197 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:52.246846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.247125 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.247399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.248113 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:52.250277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:52.260470 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1376/2750] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:52.264663 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:52.275223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ad67-fbc1-47d2-9c7e-e7204e350518 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:52.279775 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81123-29ab-4e74-88dd-80ad5bafea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:52.280517 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81123-29ab-4e74-88dd-80ad5bafea0a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.280933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81123-29ab-4e74-88dd-80ad5bafea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.281271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81123-29ab-4e74-88dd-80ad5bafea0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.282055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e81123-29ab-4e74-88dd-80ad5bafea0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:52.283638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e81123-29ab-4e74-88dd-80ad5bafea0a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:52.293490 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1375/2751] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:08:52.297517 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:52.307250 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a06629-ff71-4ba2-860e-0b245adcb7c2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:52.312361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:52.313239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.313520 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.313792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] PATH_INFO: `/v3/projects/7b923a6a03da4952bc82d0df00cfcd85/users/d26f1de050424d339edc96350160e75c/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.328571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b923a6a03da4952bc82d0df00cfcd85, user_id=d26f1de050424d339edc96350160e75c, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:52.331221 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:52.345063 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1377/2752] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:08:52 2022] PUT /identity/v3/projects/7b923a6a03da4952bc82d0df00cfcd85/users/d26f1de050424d339edc96350160e75c/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:08:52.355142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050abe48-40b7-4f51-937a-6eff8c0ad01d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.355434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050abe48-40b7-4f51-937a-6eff8c0ad01d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.355714 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050abe48-40b7-4f51-937a-6eff8c0ad01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.383798 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-050abe48-40b7-4f51-937a-6eff8c0ad01d None None] MFA Rules not processed for user `d26f1de050424d339edc96350160e75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:08:52.385008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-050abe48-40b7-4f51-937a-6eff8c0ad01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:52.462926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1376/2753] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:08:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:08:52.476002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:08:52.477957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:08:52.485795 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:08:52.543205 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f4dd-5b36-4cb3-8ed4-83d3618d2672 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:08:52.547667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263513-b5d3-4346-af39-41a916de7cfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:08:52.548313 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263513-b5d3-4346-af39-41a916de7cfd service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:08:52.548597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263513-b5d3-4346-af39-41a916de7cfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:08:52.548871 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263513-b5d3-4346-af39-41a916de7cfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:08:52.557254 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6263513-b5d3-4346-af39-41a916de7cfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:08:52.558685 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6263513-b5d3-4346-af39-41a916de7cfd service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:08:52.567922 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1378/2754] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:08:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:09:02.458889 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:02.563950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=24 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:16.315476 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050abe48-40b7-4f51-937a-6eff8c0ad01d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:16.380911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050abe48-40b7-4f51-937a-6eff8c0ad01d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:16.385828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:16.386723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:16.386985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:16.387372 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] PATH_INFO: `/v3/users/d26f1de050424d339edc96350160e75c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:16.389552 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26f1de050424d339edc96350160e75c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:16.391519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:16.447056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26f1de050424d339edc96350160e75c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:16.456752 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26f1de050424d339edc96350160e75c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:16.471333 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26f1de050424d339edc96350160e75c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:16.472056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26f1de050424d339edc96350160e75c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:16.472685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26f1de050424d339edc96350160e75c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:16.486075 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1377/2755] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:09:16 2022] DELETE /identity/v3/users/d26f1de050424d339edc96350160e75c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:09:16.492312 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:16.492852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:16.492852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:16.511004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:09:16.512207 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:16.585116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1379/2756] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:09:16.595545 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:16.597037 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:16.657819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ccbe-91d5-4910-a9a9-8d8aa30bd3bd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:16.661641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b679be0-f146-4dd7-9502-510790225b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:16.662338 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b679be0-f146-4dd7-9502-510790225b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:16.662621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b679be0-f146-4dd7-9502-510790225b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:16.662895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b679be0-f146-4dd7-9502-510790225b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:16.673605 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b679be0-f146-4dd7-9502-510790225b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:16.675057 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b679be0-f146-4dd7-9502-510790225b6a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:16.686274 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1378/2757] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:09:16.971083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:16.972543 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:16.988170 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a38c3-94eb-4031-bbce-53de3365fe89 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:16.993624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:16.994510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:16.995639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:16.995925 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] PATH_INFO: `/v3/projects/7b923a6a03da4952bc82d0df00cfcd85` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:16.997184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b923a6a03da4952bc82d0df00cfcd85)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:16.999503 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.048950 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b923a6a03da4952bc82d0df00cfcd85', 'request_id': 'req-cc6467e4-eb0d-4300-80ba-1927a61d7920'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:17.055215 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b923a6a03da4952bc82d0df00cfcd85', 'request_id': 'req-cc6467e4-eb0d-4300-80ba-1927a61d7920'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:17.060942 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b923a6a03da4952bc82d0df00cfcd85', 'request_id': 'req-cc6467e4-eb0d-4300-80ba-1927a61d7920'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:17.074151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] The token cache is being invalidate because project 7b923a6a03da4952bc82d0df00cfcd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:09:17.074447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:17.075131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cc6467e4-eb0d-4300-80ba-1927a61d7920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:17.076680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1380/2758] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:09:16 2022] DELETE /identity/v3/projects/7b923a6a03da4952bc82d0df00cfcd85 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:09:17.081822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d06f0-5dcc-4154-a07b-cf176382b725 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.082116 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d06f0-5dcc-4154-a07b-cf176382b725 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.082397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d06f0-5dcc-4154-a07b-cf176382b725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.100222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-381d06f0-5dcc-4154-a07b-cf176382b725 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:09:17.101461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-381d06f0-5dcc-4154-a07b-cf176382b725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:17.120925 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:17.166337 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1379/2759] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:09:17.174655 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea6a8d-0ff7-45d7-bc63-f04a8704f86f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.174947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea6a8d-0ff7-45d7-bc63-f04a8704f86f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.175229 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea6a8d-0ff7-45d7-bc63-f04a8704f86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.200264 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-43ea6a8d-0ff7-45d7-bc63-f04a8704f86f None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:09:17.202049 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-43ea6a8d-0ff7-45d7-bc63-f04a8704f86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:17.213219 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1381/2760] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:09:17.218424 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d06f0-5dcc-4154-a07b-cf176382b725 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:17.228387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d06f0-5dcc-4154-a07b-cf176382b725 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:17.237416 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:17.240933 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.241650 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.241936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.242677 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:09:17.242992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:17.244638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.256621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1380/2761] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:09:17.261409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea6a8d-0ff7-45d7-bc63-f04a8704f86f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:17.273071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea6a8d-0ff7-45d7-bc63-f04a8704f86f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:17.277699 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:17.278346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.278626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.278901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.280183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:17.282110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.312171 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1382/2762] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:09:17 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:09:17.316799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:17.327459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c3bd1-0ccd-4ddc-8843-3831ef983fa5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:17.385010 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:17.385969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.386256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.386535 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.387241 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:17.389733 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.410030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1381/2763] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:09:17 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:09:17.414208 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:17.425061 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f3054-842f-4b42-94c7-735861003fd2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:17.429247 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:17.429903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.430188 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.430464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.431190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:17.432789 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.443150 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1383/2764] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:09:17.447256 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:17.458034 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b18202-2801-4153-b828-5108028d4120 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:17.462874 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c997bb-f58b-4444-984b-bcc704250cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:17.463596 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c997bb-f58b-4444-984b-bcc704250cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.463828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c997bb-f58b-4444-984b-bcc704250cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.464102 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c997bb-f58b-4444-984b-bcc704250cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.464833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c997bb-f58b-4444-984b-bcc704250cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:17.466456 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c997bb-f58b-4444-984b-bcc704250cbb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.477630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1382/2765] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:09:17.481970 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:17.491915 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5f92-27e2-41ff-8590-a6b40c141f31 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:17.496645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:17.497560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.497842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.498117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] PATH_INFO: `/v3/projects/5a3285142c9946e9896cb591a0cd1062/users/05448b37f7704bc3b6928a71b110c23c/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.513608 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3285142c9946e9896cb591a0cd1062, user_id=05448b37f7704bc3b6928a71b110c23c, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:17.516816 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.532366 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1384/2766] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:09:17 2022] PUT /identity/v3/projects/5a3285142c9946e9896cb591a0cd1062/users/05448b37f7704bc3b6928a71b110c23c/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:09:17.543551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6dda-b2fe-4217-8431-82b488a00db4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.543851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6dda-b2fe-4217-8431-82b488a00db4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.544135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6dda-b2fe-4217-8431-82b488a00db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.572026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-813c6dda-b2fe-4217-8431-82b488a00db4 None None] MFA Rules not processed for user `05448b37f7704bc3b6928a71b110c23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:09:17.573297 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-813c6dda-b2fe-4217-8431-82b488a00db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:17.639991 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1383/2767] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:09:17.650792 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:17.652511 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:17.674801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:17.711702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cc79e-295e-4633-8cae-65dcf18aa18d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:17.715395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fa138-b63d-4ac8-9094-864379f9ef4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:17.715982 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fa138-b63d-4ac8-9094-864379f9ef4b service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:17.716262 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fa138-b63d-4ac8-9094-864379f9ef4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:17.716537 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fa138-b63d-4ac8-9094-864379f9ef4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:17.725267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fa138-b63d-4ac8-9094-864379f9ef4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:17.726706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fa138-b63d-4ac8-9094-864379f9ef4b service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:17.735629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1385/2768] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:09:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:09:27.636584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:27.731394 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=17 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:27.823055 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6dda-b2fe-4217-8431-82b488a00db4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:27.884558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6dda-b2fe-4217-8431-82b488a00db4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:27.889828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:27.890651 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:27.890929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:27.891199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] PATH_INFO: `/v3/users/05448b37f7704bc3b6928a71b110c23c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:27.892373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05448b37f7704bc3b6928a71b110c23c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:27.894313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:27.939988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05448b37f7704bc3b6928a71b110c23c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:27.947386 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05448b37f7704bc3b6928a71b110c23c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:27.959940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05448b37f7704bc3b6928a71b110c23c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:27.960576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05448b37f7704bc3b6928a71b110c23c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:27.961210 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05448b37f7704bc3b6928a71b110c23c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:27.973988 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1384/2769] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:09:27 2022] DELETE /identity/v3/users/05448b37f7704bc3b6928a71b110c23c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:09:27.978551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:27.978929 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:27.979241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:27.995930 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:09:27.997127 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:28.055456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1386/2770] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:09:28.065765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:28.067155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:28.124401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eefc6f-0656-47f0-9a12-c02af95c9373 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:28.128127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:28.128770 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:28.129059 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:28.129369 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:28.138655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:28.140769 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:28.150141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1385/2771] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:09:28.413595 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:28.415185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:28.430087 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16031301-f607-4653-9a68-c0d16fcb231a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:28.435406 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:28.437004 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:28.437266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:28.437551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] PATH_INFO: `/v3/projects/5a3285142c9946e9896cb591a0cd1062` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:28.438797 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3285142c9946e9896cb591a0cd1062)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:28.440736 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:28.486892 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3285142c9946e9896cb591a0cd1062', 'request_id': 'req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:28.493026 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3285142c9946e9896cb591a0cd1062', 'request_id': 'req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:28.499077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3285142c9946e9896cb591a0cd1062', 'request_id': 'req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:28.511244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] The token cache is being invalidate because project 5a3285142c9946e9896cb591a0cd1062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:09:28.511540 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:28.512131 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:09:28.513682 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1387/2772] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:09:28 2022] DELETE /identity/v3/projects/5a3285142c9946e9896cb591a0cd1062 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:09:38.146056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:38.508002 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:45.102305 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:45.104374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:45.234736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509e289-1399-4fb1-a941-21231533ee74 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:45.307427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:45.308199 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:45.308659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:45.308998 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:45.310938 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:45.329590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:45.331428 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:45.342929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1386/2773] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:09:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:09:48.325054 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:48.326489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:48.416832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7d6a3-f7a5-4e29-8c55-301d7e0169bf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:48.421748 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:48.422395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:48.422677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:48.422957 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:48.424726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:48.436169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:48.438295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:48.448149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1388/2774] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:09:55.338826 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=74 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:09:57.362255 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:09:57.363722 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:57.420951 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1ed74-6a35-4b3d-a98a-b9a841c95a99 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:09:57.424511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:09:57.425171 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:09:57.425459 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:09:57.425732 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:09:57.427441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:09:57.487828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:09:57.489303 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:09:57.498904 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1387/2775] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:09:58.443895 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=41 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:10:00.304010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:00.316114 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e808a72-86b8-48b2-9828-799c1303e363 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:00.325416 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:00.326145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:00.326494 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:00.326831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:00.329298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:00.341326 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:00.343028 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:00.353673 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1389/2776] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:10:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:10:03.409011 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:03.410670 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:03.427932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0987-a294-4656-9c1f-4dd60562d9c7 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:03.434928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:03.435531 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:03.435788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:03.436067 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:03.442926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:03.462890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:03.465430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:03.501648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ff8f8-ef63-4a44-95e9-ab82f48c28fa service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:03.508952 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:03.510371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:03.511128 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:03.511959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:03.516332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:03.554144 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:03.556028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:03.567579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1388/2777] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:10:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:10:03.568486 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:03.582275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035698-97f6-4fe0-a432-302d4592943c service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:03.605730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:03.605730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:03.605730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:03.605730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:03.606315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:03.606461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:03.607243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:03.623623 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1390/2778] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:10:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:10:03.631401 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:03.639109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:03.640384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:03.640723 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:03.656839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62722c08-2965-4559-b1c3-385c102cd367 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:03.664554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:03.664554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:03.664554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:03.664554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] PATH_INFO: `/v3/users/5068f06b1a8e47b88ba3294169545aa3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:03.668960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1389/2779] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:10:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:10:03.686154 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5068f06b1a8e47b88ba3294169545aa3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:03.688710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:03.759301 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5068f06b1a8e47b88ba3294169545aa3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:03.776314 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5068f06b1a8e47b88ba3294169545aa3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:03.777083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5068f06b1a8e47b88ba3294169545aa3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:03.777743 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5068f06b1a8e47b88ba3294169545aa3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:03.786870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5068f06b1a8e47b88ba3294169545aa3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:03.802434 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1391/2780] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:10:03 2022] DELETE /identity/v3/users/5068f06b1a8e47b88ba3294169545aa3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:10:03.985273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:04.011461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53464025-48b3-4e37-9a23-9d43679083e8 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:04.045673 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:10:04.072425 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:04.073329 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:04.073610 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:04.073868 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] PATH_INFO: `/v3/projects/ed15631acafd41528b96f2ed12c19e5b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:04.084960 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed15631acafd41528b96f2ed12c19e5b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:04.086905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:04.135982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed15631acafd41528b96f2ed12c19e5b', 'request_id': 'req-570a8d61-5c28-4a09-bb83-b01a2507f0a4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.143936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed15631acafd41528b96f2ed12c19e5b', 'request_id': 'req-570a8d61-5c28-4a09-bb83-b01a2507f0a4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.151363 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed15631acafd41528b96f2ed12c19e5b', 'request_id': 'req-570a8d61-5c28-4a09-bb83-b01a2507f0a4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.165847 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] The token cache is being invalidate because project ed15631acafd41528b96f2ed12c19e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:10:04.166127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.166787 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.168351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1390/2781] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:10:03 2022] DELETE /identity/v3/projects/ed15631acafd41528b96f2ed12c19e5b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:10:04.173138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:04.174593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:04.189765 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1a75e-3e29-4f31-8fdf-63813cbbbc89 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:04.236702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:10:04.253834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:04.256332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:04.256635 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:04.256919 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] PATH_INFO: `/v3/users/376e56dc18f14fabb7265c74a152dc69` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:04.274185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=376e56dc18f14fabb7265c74a152dc69)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:04.276179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:04.321944 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '376e56dc18f14fabb7265c74a152dc69'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.334332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '376e56dc18f14fabb7265c74a152dc69'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.335360 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '376e56dc18f14fabb7265c74a152dc69'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.336014 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '376e56dc18f14fabb7265c74a152dc69'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.346094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-33f1ef14-c859-49b2-b25b-367c8a23c4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '376e56dc18f14fabb7265c74a152dc69'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.360579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1392/2782] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:10:04 2022] DELETE /identity/v3/users/376e56dc18f14fabb7265c74a152dc69 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:10:04.543105 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:10:04.545339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:04.582519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a8d61-5c28-4a09-bb83-b01a2507f0a4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:10:04.676837 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:10:04.677417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:04.677702 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:04.677981 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] PATH_INFO: `/v3/projects/e23df072c3ad4ce6b25f813c47cb1d1f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:04.693042 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23df072c3ad4ce6b25f813c47cb1d1f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:10:04.695066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:10:04.770911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23df072c3ad4ce6b25f813c47cb1d1f', 'request_id': 'req-a173ed32-7600-4e2f-b9a2-8b38054e0657'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.773929 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:10:04.781675 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23df072c3ad4ce6b25f813c47cb1d1f', 'request_id': 'req-a173ed32-7600-4e2f-b9a2-8b38054e0657'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.794194 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23df072c3ad4ce6b25f813c47cb1d1f', 'request_id': 'req-a173ed32-7600-4e2f-b9a2-8b38054e0657'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.812383 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] The token cache is being invalidate because project e23df072c3ad4ce6b25f813c47cb1d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:10:04.812964 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.813794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:10:04.815522 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1391/2783] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:10:04 2022] DELETE /identity/v3/projects/e23df072c3ad4ce6b25f813c47cb1d1f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:10:08.127261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fb345-c102-44d0-a098-e831de717b5d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:10:08.127757 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fb345-c102-44d0-a098-e831de717b5d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:10:08.127886 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fb345-c102-44d0-a098-e831de717b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:10:08.147082 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2a0fb345-c102-44d0-a098-e831de717b5d None None] MFA Rules not processed for user `18906ec812ac49b8ba849775215c4318`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:10:08.148277 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a0fb345-c102-44d0-a098-e831de717b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:10:08.203338 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1393/2784] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:10:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:10:14.807828 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:10:18.196156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=62,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:11:37.965745 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:11:37.967132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:11:38.023374 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ed32-7600-4e2f-b9a2-8b38054e0657 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:11:38.026963 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:11:38.027604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:11:38.027886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:11:38.028168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:11:38.029952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:11:38.098250 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:11:38.099676 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:11:38.111140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1392/2785] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:11:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:11:47.977251 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fb345-c102-44d0-a098-e831de717b5d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:11:47.987212 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fb345-c102-44d0-a098-e831de717b5d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:11:47.990753 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:11:47.991479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:11:47.991762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:11:47.992041 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:11:47.993826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:11:48.007783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:11:48.009226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:11:48.020410 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1394/2786] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:11:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:11:48.105876 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=54 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:11:58.015513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:07.873576 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:07.875004 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:07.931320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b21b26-db60-44a9-a381-498cd976dfe7 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:07.934926 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:07.935571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:07.935858 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:07.936135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:07.937942 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:08.000749 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:08.002249 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:08.012157 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1393/2787] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:11.155758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:11.157319 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:11.171336 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5840c2-6f30-47cf-9425-8c23dbd45b45 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:11.176486 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:11.177407 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.177702 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.178558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] PATH_INFO: `/v3/users/18906ec812ac49b8ba849775215c4318` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.179825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18906ec812ac49b8ba849775215c4318)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:11.181820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:11.231090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18906ec812ac49b8ba849775215c4318'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.243968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18906ec812ac49b8ba849775215c4318'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.244690 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18906ec812ac49b8ba849775215c4318'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.247807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18906ec812ac49b8ba849775215c4318'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.254722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-307f77a5-e292-4857-9976-c3fafdf53725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18906ec812ac49b8ba849775215c4318'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.274865 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1395/2788] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:11 2022] DELETE /identity/v3/users/18906ec812ac49b8ba849775215c4318 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:11.447903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:11.458281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbcd9b-7ee3-4c97-84b8-46471b4ddc87 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:11.517761 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:11.518626 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.518911 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.519188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] PATH_INFO: `/v3/projects/3fcd25a6c4574894a8270e3a43af782d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.520392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fcd25a6c4574894a8270e3a43af782d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:11.522361 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:11.571661 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fcd25a6c4574894a8270e3a43af782d', 'request_id': 'req-28a9321b-d6c7-4423-86a8-80871a0f409b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.574093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:11.580375 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fcd25a6c4574894a8270e3a43af782d', 'request_id': 'req-28a9321b-d6c7-4423-86a8-80871a0f409b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.588557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fcd25a6c4574894a8270e3a43af782d', 'request_id': 'req-28a9321b-d6c7-4423-86a8-80871a0f409b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.599549 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] The token cache is being invalidate because project 3fcd25a6c4574894a8270e3a43af782d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:11.599982 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.600621 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-28a9321b-d6c7-4423-86a8-80871a0f409b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:11.602198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1394/2789] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:11 2022] DELETE /identity/v3/projects/3fcd25a6c4574894a8270e3a43af782d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:11.607118 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1266e-c3f4-4409-aa1a-38fbe122b464 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.607491 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1266e-c3f4-4409-aa1a-38fbe122b464 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.607775 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1266e-c3f4-4409-aa1a-38fbe122b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.626399 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0cb1266e-c3f4-4409-aa1a-38fbe122b464 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:11.627607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0cb1266e-c3f4-4409-aa1a-38fbe122b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:11.686477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1396/2790] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:11.694268 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94958e-0917-4a90-a385-5cfe0fbfd243 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.694558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94958e-0917-4a90-a385-5cfe0fbfd243 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.694840 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94958e-0917-4a90-a385-5cfe0fbfd243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.713654 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ed94958e-0917-4a90-a385-5cfe0fbfd243 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:11.714872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ed94958e-0917-4a90-a385-5cfe0fbfd243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:11.723594 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1395/2791] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:11.727846 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1266e-c3f4-4409-aa1a-38fbe122b464 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:11.737600 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1266e-c3f4-4409-aa1a-38fbe122b464 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:11.742695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:11.743359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.743650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.743932 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.744667 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:12:11.744978 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:11.746619 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:11.758533 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1397/2792] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:12:11.763050 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94958e-0917-4a90-a385-5cfe0fbfd243 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:11.772930 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94958e-0917-4a90-a385-5cfe0fbfd243 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:11.777611 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:11.778266 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.778551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.778827 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.779527 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:11.781445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:11.806956 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1396/2793] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:11 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:11.810987 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:11.822157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b5150-2ec3-4083-953d-57df134f1391 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:11.887571 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:11.888593 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.889027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.889510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.890606 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:11.894390 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:11.904050 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:11.920097 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1398/2794] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:11 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:11.924332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:11.934579 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ae898-e840-4ddc-acf1-c674a818eecc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:11.939167 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:11.939905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.940190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.940483 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.941230 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:11.942830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:11.953750 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1397/2795] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:11.958477 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:11.968445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edcd08-0a43-4f5a-bc9a-0b3b60f24e0e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:11.973833 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df174299-f790-47de-be01-3912d7738b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:11.974493 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df174299-f790-47de-be01-3912d7738b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:11.974782 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df174299-f790-47de-be01-3912d7738b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:11.975065 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df174299-f790-47de-be01-3912d7738b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:11.975799 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df174299-f790-47de-be01-3912d7738b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:11.977523 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df174299-f790-47de-be01-3912d7738b55 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:11.987355 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1399/2796] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:11.991397 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:12.001191 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012a98f-45f7-4e33-93c9-f0d2854983cc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:12.005765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:12.006906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:12.007187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:12.007460 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] PATH_INFO: `/v3/projects/db456be08cfa4c3b9ebf8c1e5bc396af/users/d3558eaf6ec04409819f6599a03fc5c0/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:12.022776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=db456be08cfa4c3b9ebf8c1e5bc396af, user_id=d3558eaf6ec04409819f6599a03fc5c0, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:12.025504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:12.039740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1398/2797] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:11 2022] PUT /identity/v3/projects/db456be08cfa4c3b9ebf8c1e5bc396af/users/d3558eaf6ec04409819f6599a03fc5c0/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:12.052071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc2f79-f5e9-4bb5-97bc-008684bd26ae None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:12.052365 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc2f79-f5e9-4bb5-97bc-008684bd26ae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:12.052644 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc2f79-f5e9-4bb5-97bc-008684bd26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:12.080618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-88fc2f79-f5e9-4bb5-97bc-008684bd26ae None None] MFA Rules not processed for user `d3558eaf6ec04409819f6599a03fc5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:12.081783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-88fc2f79-f5e9-4bb5-97bc-008684bd26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:12.148981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1400/2798] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:12.161878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:12.164127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:12.249243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fcb9c-c941-4e9c-b904-974673cdf8fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:12.255584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca1bb2-d19f-40ad-821d-f9e2d559b8bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f579f2db041f455786a02dcf621619ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:12.256503 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1bb2-d19f-40ad-821d-f9e2d559b8bc service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:12.256788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1bb2-d19f-40ad-821d-f9e2d559b8bc service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:12.257066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1bb2-d19f-40ad-821d-f9e2d559b8bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:12.265280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca1bb2-d19f-40ad-821d-f9e2d559b8bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:12.266869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca1bb2-d19f-40ad-821d-f9e2d559b8bc service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:12.276550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1399/2799] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:14.995354 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc2f79-f5e9-4bb5-97bc-008684bd26ae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:15.055603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc2f79-f5e9-4bb5-97bc-008684bd26ae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:15.060090 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:15.060961 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.061280 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.061559 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] PATH_INFO: `/v3/users/d3558eaf6ec04409819f6599a03fc5c0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.062760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3558eaf6ec04409819f6599a03fc5c0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:15.064693 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:15.112134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3558eaf6ec04409819f6599a03fc5c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.123956 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3558eaf6ec04409819f6599a03fc5c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.124624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3558eaf6ec04409819f6599a03fc5c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.125333 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3558eaf6ec04409819f6599a03fc5c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.129801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:15.133824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3558eaf6ec04409819f6599a03fc5c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.148202 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1401/2800] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:14 2022] DELETE /identity/v3/users/d3558eaf6ec04409819f6599a03fc5c0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:15.152704 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.153219 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.153339 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.170915 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:15.172128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:15.233122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1400/2801] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:15.243522 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:15.244954 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:15.302046 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afadd7-056a-49b8-8285-e48933415a84 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:15.305654 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cee7f1-38a1-463a-9a53-df85e61fdfec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:15.306295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee7f1-38a1-463a-9a53-df85e61fdfec service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.306578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee7f1-38a1-463a-9a53-df85e61fdfec service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.306855 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee7f1-38a1-463a-9a53-df85e61fdfec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.315904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cee7f1-38a1-463a-9a53-df85e61fdfec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:15.317359 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cee7f1-38a1-463a-9a53-df85e61fdfec service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:15.326649 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1402/2802] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:15.592588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:15.594161 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:15.602287 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:15.616822 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cae87-560a-4ca3-9e18-2bc8089cf9d3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:15.621712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:15.622542 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.622893 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.623180 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] PATH_INFO: `/v3/projects/db456be08cfa4c3b9ebf8c1e5bc396af` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.624379 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db456be08cfa4c3b9ebf8c1e5bc396af)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:15.626370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:15.673869 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db456be08cfa4c3b9ebf8c1e5bc396af', 'request_id': 'req-39354757-3818-4b32-9d95-16ff7a82c792'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.681033 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db456be08cfa4c3b9ebf8c1e5bc396af', 'request_id': 'req-39354757-3818-4b32-9d95-16ff7a82c792'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.687381 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db456be08cfa4c3b9ebf8c1e5bc396af', 'request_id': 'req-39354757-3818-4b32-9d95-16ff7a82c792'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.697947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] The token cache is being invalidate because project db456be08cfa4c3b9ebf8c1e5bc396af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:15.698239 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.698873 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-39354757-3818-4b32-9d95-16ff7a82c792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:15.700415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1401/2803] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:15 2022] DELETE /identity/v3/projects/db456be08cfa4c3b9ebf8c1e5bc396af => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:15.705821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9ea2-56fe-4733-8354-8d48df52ec6a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.706138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9ea2-56fe-4733-8354-8d48df52ec6a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.706445 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9ea2-56fe-4733-8354-8d48df52ec6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.724343 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9ded9ea2-56fe-4733-8354-8d48df52ec6a None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:15.725788 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9ded9ea2-56fe-4733-8354-8d48df52ec6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:15.784169 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1403/2804] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:15.791813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0023-0952-4222-8685-fa83a4df5095 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.792223 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0023-0952-4222-8685-fa83a4df5095 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.792504 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0023-0952-4222-8685-fa83a4df5095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.811774 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ef1b0023-0952-4222-8685-fa83a4df5095 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:15.812968 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef1b0023-0952-4222-8685-fa83a4df5095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:15.821387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1402/2805] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:15.825624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9ea2-56fe-4733-8354-8d48df52ec6a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:15.836145 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9ea2-56fe-4733-8354-8d48df52ec6a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:15.840718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:15.841413 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.841731 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.842012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.842770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:12:15.843078 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:15.844675 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:15.856351 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1404/2806] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:12:15.860736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0023-0952-4222-8685-fa83a4df5095 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:15.870493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0023-0952-4222-8685-fa83a4df5095 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:15.875445 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:15.876111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.876914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.877195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.877890 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:15.879783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:15.905832 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1403/2807] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:15 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:15.910023 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:15.920180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2660b98-b712-4cfb-88df-e542d8acc9aa admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:15.977356 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:15.977953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:15.978239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:15.978516 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:15.979209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:15.981115 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.000435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1405/2808] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:15 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:16.005120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.015642 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916b52f-81bb-4560-b4f3-e5993aedc814 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.020875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.021614 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.021859 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.022140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.022867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.024461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.034800 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1404/2809] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:12:16.038844 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.048984 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac070cfa-eace-4882-b4e4-94714c773756 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.053659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc3e3f-05bd-49a3-8b7e-eeae934a3240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.054348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc3e3f-05bd-49a3-8b7e-eeae934a3240 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.054637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc3e3f-05bd-49a3-8b7e-eeae934a3240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.054927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc3e3f-05bd-49a3-8b7e-eeae934a3240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.055641 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc3e3f-05bd-49a3-8b7e-eeae934a3240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.057257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc3e3f-05bd-49a3-8b7e-eeae934a3240 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.058575 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:16.069578 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1406/2810] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:12:16.073557 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.084008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03620bfd-fcf9-4cb0-b0a1-7a49fddc5536 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.088585 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.089502 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.089785 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.090058 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] PATH_INFO: `/v3/projects/3d8f6068f98c407daa5737da9a661d67/users/fa3016fb736c48149904254801d8d730/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.104336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8f6068f98c407daa5737da9a661d67, user_id=fa3016fb736c48149904254801d8d730, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.106989 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.119518 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1405/2811] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:16 2022] PUT /identity/v3/projects/3d8f6068f98c407daa5737da9a661d67/users/fa3016fb736c48149904254801d8d730/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:16.132482 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a7995-df60-4a7b-8d6f-115c583baffd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.132780 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a7995-df60-4a7b-8d6f-115c583baffd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.133066 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a7995-df60-4a7b-8d6f-115c583baffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.160405 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9c6a7995-df60-4a7b-8d6f-115c583baffd None None] MFA Rules not processed for user `fa3016fb736c48149904254801d8d730`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:16.161639 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c6a7995-df60-4a7b-8d6f-115c583baffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:16.226223 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1407/2812] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:12:16.230884 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.292494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02fed-239d-413d-a2d4-e206a2bebe97 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.297129 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.297810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.298581 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.298860 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.299563 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.301512 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.327792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1406/2813] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:16 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:12:16.331829 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a7995-df60-4a7b-8d6f-115c583baffd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.342638 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a7995-df60-4a7b-8d6f-115c583baffd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.399464 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.400137 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.400423 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.400701 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.401424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.403340 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.422645 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1408/2814] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:16 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:16.427128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.437387 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4d33b-0413-4062-a1fb-316e342b5522 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.442551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.443200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.443478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.443747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.444485 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.446293 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.447533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:16.460921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1407/2815] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:16.462089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.472079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9edfb-37db-4edd-a12b-9087ddc7c590 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.476722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.477637 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.477926 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.478204 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] PATH_INFO: `/v3/projects/9be60d65c43c4128b7fe8ec89c947456/users/654c7ca531ca4581aa2361bbba272e3d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.493376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be60d65c43c4128b7fe8ec89c947456, user_id=654c7ca531ca4581aa2361bbba272e3d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.495968 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.509650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1409/2816] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:16 2022] PUT /identity/v3/projects/9be60d65c43c4128b7fe8ec89c947456/users/654c7ca531ca4581aa2361bbba272e3d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:16.513603 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.523747 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b205b-5208-45b1-b831-cffe1c07e88f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.579036 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdb749-9d4d-4693-9623-316d36d15263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.579689 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdb749-9d4d-4693-9623-316d36d15263 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.579969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdb749-9d4d-4693-9623-316d36d15263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.580238 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdb749-9d4d-4693-9623-316d36d15263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.580946 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdb749-9d4d-4693-9623-316d36d15263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.582554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdb749-9d4d-4693-9623-316d36d15263 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.592207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1408/2817] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:16.596835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.607552 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3536e90-6ad2-420a-9bc6-4783805b88ef admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.612818 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.613747 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.614031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.614946 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] PATH_INFO: `/v3/projects/9be60d65c43c4128b7fe8ec89c947456/users/654c7ca531ca4581aa2361bbba272e3d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.616935 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be60d65c43c4128b7fe8ec89c947456, user_id=654c7ca531ca4581aa2361bbba272e3d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.619618 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.629996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1410/2818] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:16 2022] PUT /identity/v3/projects/9be60d65c43c4128b7fe8ec89c947456/users/654c7ca531ca4581aa2361bbba272e3d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:16.640620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.640944 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.641253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.668054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] MFA Rules not processed for user `654c7ca531ca4581aa2361bbba272e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:16.669307 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:16.736188 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1409/2819] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:16.747162 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.748783 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:16.793904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:16.811015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082e422-2a0e-4c94-9e5c-9cbe0b30762a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.814659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.815300 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.815580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.815852 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.824650 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.826275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.835867 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1411/2820] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:16.883138 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:16.884519 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:16.896940 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a2a2e-67ab-47ac-9901-5c3c01868ce8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:16.900588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93799ac2-9a7b-40fb-83e4-51d8573bf278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:16.901260 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799ac2-9a7b-40fb-83e4-51d8573bf278 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:16.901548 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799ac2-9a7b-40fb-83e4-51d8573bf278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:16.901824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799ac2-9a7b-40fb-83e4-51d8573bf278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:16.957584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93799ac2-9a7b-40fb-83e4-51d8573bf278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:16.959044 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93799ac2-9a7b-40fb-83e4-51d8573bf278 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:16.967810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1410/2821] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:20.224361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:20.287216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed336de-4445-4d23-8df9-271ce57bd7fe service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:20.291761 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:20.292621 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:20.292903 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:20.293294 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] PATH_INFO: `/v3/users/fa3016fb736c48149904254801d8d730` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:20.294480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3016fb736c48149904254801d8d730)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:20.296389 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:20.341763 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3016fb736c48149904254801d8d730'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.353887 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3016fb736c48149904254801d8d730'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.354560 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3016fb736c48149904254801d8d730'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.355190 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3016fb736c48149904254801d8d730'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.362461 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3016fb736c48149904254801d8d730'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.374442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1412/2822] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:20 2022] DELETE /identity/v3/users/fa3016fb736c48149904254801d8d730 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:20.379440 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:20.379952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:20.380066 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:20.397730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:20.398932 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:20.407941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:20.465206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1411/2823] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:20.474854 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:20.476298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:20.536109 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327c1a8-b893-4d6d-8017-9b5f34a7a7db admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:20.539890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:20.540544 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:20.540830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:20.541108 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:20.550447 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:20.551891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:20.561613 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1413/2824] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:20.836090 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:20.837584 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:20.849589 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65faf-603d-49b0-9e90-0971585f399e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:20.855251 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:20.856228 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:20.856570 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:20.856928 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] PATH_INFO: `/v3/projects/3d8f6068f98c407daa5737da9a661d67` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:20.865959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8f6068f98c407daa5737da9a661d67)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:20.867899 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:20.914538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8f6068f98c407daa5737da9a661d67', 'request_id': 'req-549e178c-991a-49f4-8aa8-b771e7c9b776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.921791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8f6068f98c407daa5737da9a661d67', 'request_id': 'req-549e178c-991a-49f4-8aa8-b771e7c9b776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.928662 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8f6068f98c407daa5737da9a661d67', 'request_id': 'req-549e178c-991a-49f4-8aa8-b771e7c9b776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.939506 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] The token cache is being invalidate because project 3d8f6068f98c407daa5737da9a661d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:20.939795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.940432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:20.942008 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1412/2825] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:20 2022] DELETE /identity/v3/projects/3d8f6068f98c407daa5737da9a661d67 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:20.945971 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:20.950777 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:20.956077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab955-2a30-4820-8a65-6002407dae93 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.017179 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.017963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.018241 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.018510 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] PATH_INFO: `/v3/users/654c7ca531ca4581aa2361bbba272e3d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.019680 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=654c7ca531ca4581aa2361bbba272e3d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.021638 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.067130 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654c7ca531ca4581aa2361bbba272e3d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.078808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654c7ca531ca4581aa2361bbba272e3d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.079487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654c7ca531ca4581aa2361bbba272e3d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.080098 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654c7ca531ca4581aa2361bbba272e3d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.096112 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fdfccd12-33cf-4434-820e-9a96ae449f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654c7ca531ca4581aa2361bbba272e3d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.109134 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1414/2826] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:20 2022] DELETE /identity/v3/users/654c7ca531ca4581aa2361bbba272e3d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:21.381207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:21.382634 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:21.403179 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e178c-991a-49f4-8aa8-b771e7c9b776 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.460190 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.461053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.461370 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.461645 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] PATH_INFO: `/v3/projects/9be60d65c43c4128b7fe8ec89c947456` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.462819 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be60d65c43c4128b7fe8ec89c947456)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.464712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.505382 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:21.512701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be60d65c43c4128b7fe8ec89c947456', 'request_id': 'req-8e65b9f0-2779-4343-94a3-45aecb6670dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.521040 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be60d65c43c4128b7fe8ec89c947456', 'request_id': 'req-8e65b9f0-2779-4343-94a3-45aecb6670dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.527222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be60d65c43c4128b7fe8ec89c947456', 'request_id': 'req-8e65b9f0-2779-4343-94a3-45aecb6670dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.537861 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] The token cache is being invalidate because project 9be60d65c43c4128b7fe8ec89c947456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:21.538149 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.538765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e65b9f0-2779-4343-94a3-45aecb6670dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:21.540281 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1413/2827] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:21 2022] DELETE /identity/v3/projects/9be60d65c43c4128b7fe8ec89c947456 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:21.545271 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc5856-e426-4555-8a69-fb4fe5dd8e3c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.545760 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc5856-e426-4555-8a69-fb4fe5dd8e3c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.545870 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc5856-e426-4555-8a69-fb4fe5dd8e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.564429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-affc5856-e426-4555-8a69-fb4fe5dd8e3c None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:21.565607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-affc5856-e426-4555-8a69-fb4fe5dd8e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:21.625101 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1415/2828] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:21.633417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6743b-a855-4df2-850c-2b0d2b6d48e6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.633708 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6743b-a855-4df2-850c-2b0d2b6d48e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.633996 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6743b-a855-4df2-850c-2b0d2b6d48e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.652712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b9c6743b-a855-4df2-850c-2b0d2b6d48e6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:21.653992 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9c6743b-a855-4df2-850c-2b0d2b6d48e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:21.662464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1414/2829] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:21.666636 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc5856-e426-4555-8a69-fb4fe5dd8e3c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:21.676243 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc5856-e426-4555-8a69-fb4fe5dd8e3c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.680784 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.681451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.681738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.682012 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.682730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:12:21.683031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.684609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.699969 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1416/2830] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:12:21.706601 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6743b-a855-4df2-850c-2b0d2b6d48e6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:21.716520 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6743b-a855-4df2-850c-2b0d2b6d48e6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.721688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.722336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.722616 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.722888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.723588 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.725510 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.750164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1415/2831] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:21 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:21.754226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:21.765268 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bef089-db25-49fd-bd0e-1f78120af975 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.770570 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:21.826216 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.826798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.827079 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.827349 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.828037 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.830565 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.850253 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1417/2832] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:21 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:21.854225 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:21.864313 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade52385-2c1b-458c-b76f-53bd779e93f7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.868811 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.869493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.869776 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.870049 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.870763 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.872346 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.883248 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1416/2833] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:21.887071 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:21.897597 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834ee49-f8b2-4c00-b8a2-e6df001a37e0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.902192 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ccde3-ced4-4594-b86a-191719532657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.902849 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ccde3-ced4-4594-b86a-191719532657 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.903136 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ccde3-ced4-4594-b86a-191719532657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.903415 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ccde3-ced4-4594-b86a-191719532657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.904142 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ccde3-ced4-4594-b86a-191719532657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.905778 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ccde3-ced4-4594-b86a-191719532657 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.915770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1418/2834] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:21.921501 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:21.936099 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf68fd-4f6c-4ab6-9e76-6b37a31ad431 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:21.941202 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:21.942081 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.942358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.942630 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] PATH_INFO: `/v3/projects/800e9788082146f780f5b0a73b4805c8/users/d9c588c3c9f047fb81f12ca1dee5e9cf/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:21.958595 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] RBAC: Authorizing `identity:create_grant(project_id=800e9788082146f780f5b0a73b4805c8, user_id=d9c588c3c9f047fb81f12ca1dee5e9cf, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:21.961258 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:21.973783 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1417/2835] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:21 2022] PUT /identity/v3/projects/800e9788082146f780f5b0a73b4805c8/users/d9c588c3c9f047fb81f12ca1dee5e9cf/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:21.985226 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41117c9-c257-41b0-b16d-d353298aa273 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:21.985527 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41117c9-c257-41b0-b16d-d353298aa273 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:21.985811 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41117c9-c257-41b0-b16d-d353298aa273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.012722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b41117c9-c257-41b0-b16d-d353298aa273 None None] MFA Rules not processed for user `d9c588c3c9f047fb81f12ca1dee5e9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:22.013977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b41117c9-c257-41b0-b16d-d353298aa273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:22.080498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1419/2836] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:12:22.084959 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.148727 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e285e2e-223a-45e3-b7d1-1563dc8c286b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.153475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.154140 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.154427 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.154701 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.155395 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.157462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.183813 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1418/2837] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:22 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:12:22.187798 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41117c9-c257-41b0-b16d-d353298aa273 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.199750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41117c9-c257-41b0-b16d-d353298aa273 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.256409 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.257083 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.257398 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.257677 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.258370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.260287 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.280077 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1420/2838] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:22 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:22.284039 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.294189 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67b37-e140-4c17-a362-905ea582d96b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.299200 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.299852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.300131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.300404 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.301178 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.302789 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.314308 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1419/2839] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:22.318157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.328084 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba41d-8e10-4f15-b6d9-90f5348f216c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.333480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.334629 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.335119 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.335396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] PATH_INFO: `/v3/projects/16cc484301f14ce18fed320fd09e9a89/users/ee905de513694ad1b3d0b72849f740d6/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.337807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:22.352322 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cc484301f14ce18fed320fd09e9a89, user_id=ee905de513694ad1b3d0b72849f740d6, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.355021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.367830 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1421/2840] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:22 2022] PUT /identity/v3/projects/16cc484301f14ce18fed320fd09e9a89/users/ee905de513694ad1b3d0b72849f740d6/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:22.371791 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.382365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7ecb6-0119-450c-abaf-abae04480b49 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.418358 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:22.438538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652ecea-483a-4b8a-aec8-8f67fd1b4b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.439694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652ecea-483a-4b8a-aec8-8f67fd1b4b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.439972 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652ecea-483a-4b8a-aec8-8f67fd1b4b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.440243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652ecea-483a-4b8a-aec8-8f67fd1b4b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.440954 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652ecea-483a-4b8a-aec8-8f67fd1b4b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.442568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652ecea-483a-4b8a-aec8-8f67fd1b4b21 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.452153 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1420/2841] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:22.456478 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.466745 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910e2ed-faa9-4e9b-bb65-3082f7b7c769 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.473909 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.475448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.475985 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.476536 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] PATH_INFO: `/v3/projects/16cc484301f14ce18fed320fd09e9a89/users/ee905de513694ad1b3d0b72849f740d6/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.479074 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cc484301f14ce18fed320fd09e9a89, user_id=ee905de513694ad1b3d0b72849f740d6, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.481738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.492786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1422/2842] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:22 2022] PUT /identity/v3/projects/16cc484301f14ce18fed320fd09e9a89/users/ee905de513694ad1b3d0b72849f740d6/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:12:22.504739 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76ad38-b205-4e96-b9b9-73545074e22b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.505025 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76ad38-b205-4e96-b9b9-73545074e22b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.505334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76ad38-b205-4e96-b9b9-73545074e22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.531324 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ca76ad38-b205-4e96-b9b9-73545074e22b None None] MFA Rules not processed for user `ee905de513694ad1b3d0b72849f740d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:22.533026 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca76ad38-b205-4e96-b9b9-73545074e22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:22.605496 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1421/2843] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:22.617296 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.619038 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:22.675772 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75aed86-3e8b-4905-ad77-0eb038222bb2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.679455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.680093 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.680381 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.680660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.689592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.691031 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.700476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1423/2844] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:22.771734 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76ad38-b205-4e96-b9b9-73545074e22b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:22.781417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76ad38-b205-4e96-b9b9-73545074e22b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:22.785564 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0dcad-8177-4d6d-9ce1-eb1f3b793a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:22.786206 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0dcad-8177-4d6d-9ce1-eb1f3b793a84 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:22.786491 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0dcad-8177-4d6d-9ce1-eb1f3b793a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:22.786765 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0dcad-8177-4d6d-9ce1-eb1f3b793a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:22.842656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0dcad-8177-4d6d-9ce1-eb1f3b793a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:22.844108 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0dcad-8177-4d6d-9ce1-eb1f3b793a84 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:22.853969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1422/2845] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:23.117966 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:23.180776 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5faf7-f904-4a3e-9d87-46806423fca3 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:23.185574 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:23.186459 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:23.186744 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:23.187025 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] PATH_INFO: `/v3/users/d9c588c3c9f047fb81f12ca1dee5e9cf` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:23.188224 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c588c3c9f047fb81f12ca1dee5e9cf)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:23.190762 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:23.238062 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c588c3c9f047fb81f12ca1dee5e9cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.249937 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c588c3c9f047fb81f12ca1dee5e9cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.250627 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c588c3c9f047fb81f12ca1dee5e9cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.251769 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c588c3c9f047fb81f12ca1dee5e9cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.261890 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c588c3c9f047fb81f12ca1dee5e9cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.264217 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:23.280556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1424/2846] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:23 2022] DELETE /identity/v3/users/d9c588c3c9f047fb81f12ca1dee5e9cf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:23.287556 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:23.287906 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:23.288233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:23.310931 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:23.312575 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:23.368738 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:23.377493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1423/2847] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:23.387489 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:23.389089 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:23.448710 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2067a-0ff0-4f41-b4c2-6b6af0f1011e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:23.452558 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:23.453244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:23.453561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:23.453842 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:23.463288 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:23.464729 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:23.474934 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1425/2848] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:23.738597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:23.740128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:23.752590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e46e74-b7f8-4cb8-a267-fea47564ff3e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:23.758694 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:23.759683 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:23.760038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:23.760380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] PATH_INFO: `/v3/projects/800e9788082146f780f5b0a73b4805c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:23.761685 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=800e9788082146f780f5b0a73b4805c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:23.763718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:23.811288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800e9788082146f780f5b0a73b4805c8', 'request_id': 'req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.819056 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800e9788082146f780f5b0a73b4805c8', 'request_id': 'req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.825687 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800e9788082146f780f5b0a73b4805c8', 'request_id': 'req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.836762 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] The token cache is being invalidate because project 800e9788082146f780f5b0a73b4805c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:23.837052 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.837740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.839322 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1424/2849] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:23 2022] DELETE /identity/v3/projects/800e9788082146f780f5b0a73b4805c8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:23.843479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:23.854771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf2b3-d009-4a9a-a69b-dc679902426e service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:23.912456 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:23.913266 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:23.913549 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:23.913823 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] PATH_INFO: `/v3/users/ee905de513694ad1b3d0b72849f740d6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:23.915008 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee905de513694ad1b3d0b72849f740d6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:23.916924 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:23.966900 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee905de513694ad1b3d0b72849f740d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.979487 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee905de513694ad1b3d0b72849f740d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.980173 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee905de513694ad1b3d0b72849f740d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.980786 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee905de513694ad1b3d0b72849f740d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:23.988901 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0e00edd-470c-416a-9e7e-e2519bd3a790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee905de513694ad1b3d0b72849f740d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:24.001963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1426/2850] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:23 2022] DELETE /identity/v3/users/ee905de513694ad1b3d0b72849f740d6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:24.281688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.283150 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:24.296232 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6d0c-cfad-4196-8dfc-2ea7862193a2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:24.354128 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:24.354994 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.355279 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.355659 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] PATH_INFO: `/v3/projects/16cc484301f14ce18fed320fd09e9a89` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.356851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cc484301f14ce18fed320fd09e9a89)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:24.358852 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:24.405024 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cc484301f14ce18fed320fd09e9a89', 'request_id': 'req-df6c5716-431f-4596-a3be-d01af224053e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:24.412273 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cc484301f14ce18fed320fd09e9a89', 'request_id': 'req-df6c5716-431f-4596-a3be-d01af224053e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:24.418553 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cc484301f14ce18fed320fd09e9a89', 'request_id': 'req-df6c5716-431f-4596-a3be-d01af224053e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:24.430113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] The token cache is being invalidate because project 16cc484301f14ce18fed320fd09e9a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:24.430406 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:24.431029 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df6c5716-431f-4596-a3be-d01af224053e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:24.432554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1425/2851] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:24 2022] DELETE /identity/v3/projects/16cc484301f14ce18fed320fd09e9a89 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:24.438315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45a4b-716b-4c4e-9f59-3320d6b87ed6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.438515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45a4b-716b-4c4e-9f59-3320d6b87ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.439298 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45a4b-716b-4c4e-9f59-3320d6b87ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.458491 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dfc45a4b-716b-4c4e-9f59-3320d6b87ed6 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:24.459706 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dfc45a4b-716b-4c4e-9f59-3320d6b87ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:24.468210 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:24.523453 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1427/2852] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:24.531127 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd95f-941a-44c6-8804-7fc73632aaa7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.531417 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd95f-941a-44c6-8804-7fc73632aaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.531697 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd95f-941a-44c6-8804-7fc73632aaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.550742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1d6fd95f-941a-44c6-8804-7fc73632aaa7 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:24.551941 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d6fd95f-941a-44c6-8804-7fc73632aaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:24.559863 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1426/2853] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:24.565880 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45a4b-716b-4c4e-9f59-3320d6b87ed6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.576058 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45a4b-716b-4c4e-9f59-3320d6b87ed6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:24.580826 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:24.581524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.581805 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.582085 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.582831 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:12:24.583138 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:24.584726 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:24.596260 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1428/2854] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:12:24.601706 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd95f-941a-44c6-8804-7fc73632aaa7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.611538 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd95f-941a-44c6-8804-7fc73632aaa7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:24.616253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:24.616897 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.617222 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.617507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.618213 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:24.620132 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:24.623454 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:24.651349 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1427/2855] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:24 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:24.655808 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.666455 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f582ca-118b-48fe-be8d-5866f6222aa6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:24.726376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:24.727045 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.727328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.727604 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.728297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:24.730239 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:24.750348 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1429/2856] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:24 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:24.755624 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.766432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399e0c-f2b7-4efa-8638-cc8f842b180d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:24.771086 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:24.771746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.772032 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.772306 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.773030 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:24.774655 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:24.786740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1428/2857] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:24.790815 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.802249 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e2d09-becb-4fc3-8845-4de231817d53 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:24.807518 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d71381-6a6d-44e8-b88f-9bcf8ba173fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:24.808184 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d71381-6a6d-44e8-b88f-9bcf8ba173fb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.809498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d71381-6a6d-44e8-b88f-9bcf8ba173fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.809498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d71381-6a6d-44e8-b88f-9bcf8ba173fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.809498 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d71381-6a6d-44e8-b88f-9bcf8ba173fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:24.811099 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d71381-6a6d-44e8-b88f-9bcf8ba173fb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:24.821395 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1430/2858] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:24.825468 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.835319 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f573-812e-4171-bc7a-a345a0819310 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:24.840922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:24.841833 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.842122 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.842401 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] PATH_INFO: `/v3/projects/2e171a623e5d44379c21e187ce1ff97c/users/78b611e420394c3b98cc9b072180bbdc/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.858077 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e171a623e5d44379c21e187ce1ff97c, user_id=78b611e420394c3b98cc9b072180bbdc, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:24.861479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:24.874493 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1429/2859] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:24 2022] PUT /identity/v3/projects/2e171a623e5d44379c21e187ce1ff97c/users/78b611e420394c3b98cc9b072180bbdc/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:24.886185 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45957232-aada-4b14-b594-be1b83e3402c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:24.886711 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45957232-aada-4b14-b594-be1b83e3402c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:24.886819 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45957232-aada-4b14-b594-be1b83e3402c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:24.914117 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-45957232-aada-4b14-b594-be1b83e3402c None None] MFA Rules not processed for user `78b611e420394c3b98cc9b072180bbdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:24.915335 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-45957232-aada-4b14-b594-be1b83e3402c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:24.978834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1431/2860] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:12:24.990498 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:24.992232 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:25.055436 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d2083-fe83-4e35-a068-3370e91e4244 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:25.059263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e5a1f-0564-4bcf-b4f8-a5a93617e1c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:25.059914 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e5a1f-0564-4bcf-b4f8-a5a93617e1c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.060201 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e5a1f-0564-4bcf-b4f8-a5a93617e1c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.060478 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e5a1f-0564-4bcf-b4f8-a5a93617e1c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.069186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e5a1f-0564-4bcf-b4f8-a5a93617e1c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:25.070638 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e5a1f-0564-4bcf-b4f8-a5a93617e1c0 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:25.080146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1430/2861] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:25.101419 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45957232-aada-4b14-b594-be1b83e3402c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:25.116393 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:25.176664 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45957232-aada-4b14-b594-be1b83e3402c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:25.181402 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:25.182274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.182554 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.182827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] PATH_INFO: `/v3/users/78b611e420394c3b98cc9b072180bbdc` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.184176 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b611e420394c3b98cc9b072180bbdc)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:25.186116 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:25.234660 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b611e420394c3b98cc9b072180bbdc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.246839 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b611e420394c3b98cc9b072180bbdc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.247521 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b611e420394c3b98cc9b072180bbdc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.248133 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b611e420394c3b98cc9b072180bbdc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.256648 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b611e420394c3b98cc9b072180bbdc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.269657 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1432/2862] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:25 2022] DELETE /identity/v3/users/78b611e420394c3b98cc9b072180bbdc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:25.274226 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.274524 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.274801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.293927 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:25.295137 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:25.354953 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1431/2863] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:12:25.364524 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:25.366030 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:25.426369 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17708e97-7bf1-49d8-a6ec-50e62d003d73 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:25.430151 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77a46f-3587-4abc-8a9f-a80874f21032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:25.430875 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a46f-3587-4abc-8a9f-a80874f21032 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.431157 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a46f-3587-4abc-8a9f-a80874f21032 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.431448 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a46f-3587-4abc-8a9f-a80874f21032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.441068 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77a46f-3587-4abc-8a9f-a80874f21032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:25.442566 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77a46f-3587-4abc-8a9f-a80874f21032 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:25.455697 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1433/2864] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:25.723730 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:25.725430 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:25.738952 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4e88e-ff08-4a4d-b7d4-8ae33a2c3a0b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:25.743712 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:25.745364 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.745657 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.745936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] PATH_INFO: `/v3/projects/2e171a623e5d44379c21e187ce1ff97c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.747137 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e171a623e5d44379c21e187ce1ff97c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:25.749131 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:25.795568 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:25.796980 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e171a623e5d44379c21e187ce1ff97c', 'request_id': 'req-da5e76bd-fab7-4d60-81f1-149faaf788bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.804271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e171a623e5d44379c21e187ce1ff97c', 'request_id': 'req-da5e76bd-fab7-4d60-81f1-149faaf788bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.811666 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e171a623e5d44379c21e187ce1ff97c', 'request_id': 'req-da5e76bd-fab7-4d60-81f1-149faaf788bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.822332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] The token cache is being invalidate because project 2e171a623e5d44379c21e187ce1ff97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:25.822620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.823253 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da5e76bd-fab7-4d60-81f1-149faaf788bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:25.824788 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1432/2865] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:25 2022] DELETE /identity/v3/projects/2e171a623e5d44379c21e187ce1ff97c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:25.830284 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8fba9-a852-466b-b46a-13e1a57e8762 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.830573 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8fba9-a852-466b-b46a-13e1a57e8762 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.830847 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8fba9-a852-466b-b46a-13e1a57e8762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.848186 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f4a8fba9-a852-466b-b46a-13e1a57e8762 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:25.849429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f4a8fba9-a852-466b-b46a-13e1a57e8762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:25.876496 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:25.912435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1434/2866] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:25.920147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507de10-883a-46ed-8c50-e7f3f54bd2fc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.920438 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507de10-883a-46ed-8c50-e7f3f54bd2fc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.920720 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507de10-883a-46ed-8c50-e7f3f54bd2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.940168 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c507de10-883a-46ed-8c50-e7f3f54bd2fc None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:25.941415 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c507de10-883a-46ed-8c50-e7f3f54bd2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:25.949299 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1433/2867] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:25.953996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8fba9-a852-466b-b46a-13e1a57e8762 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:25.963947 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8fba9-a852-466b-b46a-13e1a57e8762 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:25.968607 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:25.969302 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:25.969612 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:25.969888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:25.970609 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:12:25.970914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:25.972504 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:25.984373 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1435/2868] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:12:25.989270 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507de10-883a-46ed-8c50-e7f3f54bd2fc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:25.998523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507de10-883a-46ed-8c50-e7f3f54bd2fc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.003895 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.004550 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.004835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.005115 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.006133 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.008031 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.034639 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1434/2869] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:25 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:26.038750 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.049081 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ad8bb-cc98-4e50-b5f6-a5aad0dc5b38 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.108708 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.109344 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.109628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.109904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.111149 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.113107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.136904 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1436/2870] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:26 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:26.141360 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.151772 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be130522-2be0-474a-bcef-d8e00b479bc7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.156433 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.157091 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.157407 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.157688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.158419 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.160198 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.170795 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1435/2871] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:26.175856 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.186770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015177f7-fb17-4142-bfea-b035f0f753c6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.192129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eac132-2b2a-49ad-89b1-6fbdc55c3549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.192804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eac132-2b2a-49ad-89b1-6fbdc55c3549 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.193094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eac132-2b2a-49ad-89b1-6fbdc55c3549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.193400 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eac132-2b2a-49ad-89b1-6fbdc55c3549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.194129 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eac132-2b2a-49ad-89b1-6fbdc55c3549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.196172 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eac132-2b2a-49ad-89b1-6fbdc55c3549 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.206158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1437/2872] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:26.210482 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.220351 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50908efd-910f-4c59-b2ee-e248f95446f3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.225001 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.225888 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.226164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.226432 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] PATH_INFO: `/v3/projects/d5a73d0bb2b04c6594ee945a05c627ea/users/09b58220df6c43ce8f121da8d9e6a2ca/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.241536 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a73d0bb2b04c6594ee945a05c627ea, user_id=09b58220df6c43ce8f121da8d9e6a2ca, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.244155 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.258882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1436/2873] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:26 2022] PUT /identity/v3/projects/d5a73d0bb2b04c6594ee945a05c627ea/users/09b58220df6c43ce8f121da8d9e6a2ca/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:26.269528 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34964f38-f1f4-4657-b792-e9c9c6a07437 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.269827 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34964f38-f1f4-4657-b792-e9c9c6a07437 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.270110 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34964f38-f1f4-4657-b792-e9c9c6a07437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.297771 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-34964f38-f1f4-4657-b792-e9c9c6a07437 None None] MFA Rules not processed for user `09b58220df6c43ce8f121da8d9e6a2ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:26.299551 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-34964f38-f1f4-4657-b792-e9c9c6a07437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:26.362977 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1438/2874] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:26.367521 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.431523 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16673c6-460c-4fd4-b831-1b4ce0276e2f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.437985 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.439356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.439356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.439356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.439921 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.441882 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.472582 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1437/2875] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:26 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:26.476878 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34964f38-f1f4-4657-b792-e9c9c6a07437 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.487981 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34964f38-f1f4-4657-b792-e9c9c6a07437 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.512624 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:26.545707 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.546371 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.546651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.546923 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.547615 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.550156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.569531 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1439/2876] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:26 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:26.573571 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.587824 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc6074-aed9-43c7-bd6e-7373e740c79e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.594302 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.595470 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.595809 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.596135 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.596875 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.598558 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.609997 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1438/2877] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:26.614700 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.624916 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d01ff-7d3d-43b9-b3d2-39f20ca7f492 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.629718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864026b-5e3a-4094-bd78-8e3bd7dfabfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.630396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864026b-5e3a-4094-bd78-8e3bd7dfabfb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.630963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864026b-5e3a-4094-bd78-8e3bd7dfabfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.631244 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864026b-5e3a-4094-bd78-8e3bd7dfabfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.631974 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864026b-5e3a-4094-bd78-8e3bd7dfabfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.633643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864026b-5e3a-4094-bd78-8e3bd7dfabfb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.644388 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1440/2878] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:12:26.648551 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.659280 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957af6bb-020c-497f-8c0d-ec5674567233 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.663878 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.664758 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.665062 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.665441 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] PATH_INFO: `/v3/projects/53ce20ef5bcc4a0486762d6591ab5b44/users/f8ee0cde267441f2ba7a0938a271a2e2/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.680095 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ce20ef5bcc4a0486762d6591ab5b44, user_id=f8ee0cde267441f2ba7a0938a271a2e2, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.682768 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.695494 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1439/2879] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:26 2022] PUT /identity/v3/projects/53ce20ef5bcc4a0486762d6591ab5b44/users/f8ee0cde267441f2ba7a0938a271a2e2/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:12:26.755975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec86c3c-457c-43e7-b07f-8fa031ba9b8e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.756297 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec86c3c-457c-43e7-b07f-8fa031ba9b8e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.756569 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec86c3c-457c-43e7-b07f-8fa031ba9b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.793534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bec86c3c-457c-43e7-b07f-8fa031ba9b8e None None] MFA Rules not processed for user `f8ee0cde267441f2ba7a0938a271a2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:26.794758 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bec86c3c-457c-43e7-b07f-8fa031ba9b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:26.858069 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1441/2880] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:26.862849 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.888106 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:26.926644 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64666116-e050-4167-b488-f30092ebe01d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:26.931275 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:26.931936 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:26.932224 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:26.932507 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:26.933236 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:26.935142 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:26.960284 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1440/2881] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:26 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:12:26.964454 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec86c3c-457c-43e7-b07f-8fa031ba9b8e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:26.976257 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec86c3c-457c-43e7-b07f-8fa031ba9b8e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:27.033019 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:27.033718 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:27.034001 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:27.034275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:27.034953 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:27.036864 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:27.056299 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1442/2882] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:26 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:27.060462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:27.071464 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6109dc0-3bb5-46eb-935f-9c70bc7d0162 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:27.075947 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:27.076597 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:27.076879 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:27.077174 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:27.077901 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:27.080038 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:27.091320 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1441/2883] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:27.095386 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:27.105361 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e59aff-3189-4d6c-90e4-0897a5199e46 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:27.110021 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:27.110914 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:27.111201 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:27.111480 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] PATH_INFO: `/v3/projects/4a982f70e39b4484b1dc568cd9318683/users/96eae654e7df4d54850b02e74a1c2f0d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:27.126730 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a982f70e39b4484b1dc568cd9318683, user_id=96eae654e7df4d54850b02e74a1c2f0d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:27.129451 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:27.138989 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:27.144603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1443/2884] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:27 2022] PUT /identity/v3/projects/4a982f70e39b4484b1dc568cd9318683/users/96eae654e7df4d54850b02e74a1c2f0d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:27.148480 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:27.158420 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e68a1e-a54b-47ea-8244-7ad5bdb7698a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:27.217691 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34c61f-ae47-42a3-aeb0-58fbee65a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:27.218359 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34c61f-ae47-42a3-aeb0-58fbee65a8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:27.219071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34c61f-ae47-42a3-aeb0-58fbee65a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:27.219071 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34c61f-ae47-42a3-aeb0-58fbee65a8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:27.219598 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34c61f-ae47-42a3-aeb0-58fbee65a8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:27.221217 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34c61f-ae47-42a3-aeb0-58fbee65a8f9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:27.231028 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1442/2885] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:27.235158 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:27.245232 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0f760-5669-4b2c-b542-75d328a092ff admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:27.249721 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:27.250594 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:27.251367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:27.251643 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] PATH_INFO: `/v3/projects/4a982f70e39b4484b1dc568cd9318683/users/96eae654e7df4d54850b02e74a1c2f0d/roles/2e9eb416245242c8aa082ba62b6ff1b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:27.253585 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a982f70e39b4484b1dc568cd9318683, user_id=96eae654e7df4d54850b02e74a1c2f0d, role_id=2e9eb416245242c8aa082ba62b6ff1b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:27.256468 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:27.267139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1444/2886] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:27 2022] PUT /identity/v3/projects/4a982f70e39b4484b1dc568cd9318683/users/96eae654e7df4d54850b02e74a1c2f0d/roles/2e9eb416245242c8aa082ba62b6ff1b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:27.277620 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:27.277919 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:27.278336 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:27.305469 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] MFA Rules not processed for user `96eae654e7df4d54850b02e74a1c2f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:27.306679 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:27.377564 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1443/2887] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:27.390479 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:27.392695 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:27.456806 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6794e-39c4-453e-ab96-b3dd0ce0357a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:27.460579 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:27.461245 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:27.461529 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:27.461801 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:27.518422 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:27.519876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:27.528874 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1445/2888] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:29.014053 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:29.015437 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:29.028902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41437347-b049-4df9-99ee-1c1f4a2e0392 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:29.033113 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:29.033794 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:29.034078 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:29.034854 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:29.048373 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:29.097566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:29.099380 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:29.108566 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1444/2889] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:29.685295 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:29.694292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ed172-ff0b-4e27-a308-f9d4c7bcb812 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:29.698959 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2094e6d-fb2f-48f7-a417-1b37088d77e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:29.699605 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2094e6d-fb2f-48f7-a417-1b37088d77e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:29.699888 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2094e6d-fb2f-48f7-a417-1b37088d77e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:29.700165 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2094e6d-fb2f-48f7-a417-1b37088d77e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:29.713144 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2094e6d-fb2f-48f7-a417-1b37088d77e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:29.714628 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2094e6d-fb2f-48f7-a417-1b37088d77e6 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:29.724891 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1446/2890] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:33.590461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:33.654392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e2ed-dd47-4f39-897c-c5b74418681c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:33.661736 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:33.662622 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:33.662905 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:33.663184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] PATH_INFO: `/v3/users/09b58220df6c43ce8f121da8d9e6a2ca` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:33.664392 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b58220df6c43ce8f121da8d9e6a2ca)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:33.666356 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:33.713590 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b58220df6c43ce8f121da8d9e6a2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:33.722403 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b58220df6c43ce8f121da8d9e6a2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:33.734810 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b58220df6c43ce8f121da8d9e6a2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:33.735511 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b58220df6c43ce8f121da8d9e6a2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:33.736103 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b58220df6c43ce8f121da8d9e6a2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:33.749227 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1445/2891] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:33 2022] DELETE /identity/v3/users/09b58220df6c43ce8f121da8d9e6a2ca => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:33.753713 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:33.754189 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:33.754293 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:33.775614 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:33.776807 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:33.837949 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1447/2892] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:33.847903 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:33.849688 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:33.899434 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:33.918969 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72177be-7062-4490-aee7-8fc99eb76754 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:33.923209 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:33.923856 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:33.924141 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:33.924414 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:33.933792 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:33.935231 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:33.947120 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1446/2893] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:12:34.221738 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:34.223328 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:34.240824 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1e828-da56-4e71-ae12-5c1033b55389 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:34.247404 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:34.248820 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:34.249346 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:34.249663 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] PATH_INFO: `/v3/projects/d5a73d0bb2b04c6594ee945a05c627ea` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:34.251476 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a73d0bb2b04c6594ee945a05c627ea)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:34.254430 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:34.285963 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:34.310286 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a73d0bb2b04c6594ee945a05c627ea', 'request_id': 'req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.316414 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a73d0bb2b04c6594ee945a05c627ea', 'request_id': 'req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.322139 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a73d0bb2b04c6594ee945a05c627ea', 'request_id': 'req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.335267 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] The token cache is being invalidate because project d5a73d0bb2b04c6594ee945a05c627ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:34.335555 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.336183 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.337814 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1448/2894] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:34 2022] DELETE /identity/v3/projects/d5a73d0bb2b04c6594ee945a05c627ea => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:34.343186 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:34.359263 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3209358-1096-4b8b-973d-bee4c08b5dc4 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:34.418207 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:34.419093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:34.419475 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:34.419641 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] PATH_INFO: `/v3/users/f8ee0cde267441f2ba7a0938a271a2e2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:34.420830 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ee0cde267441f2ba7a0938a271a2e2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:34.422801 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:34.476923 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ee0cde267441f2ba7a0938a271a2e2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.485461 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ee0cde267441f2ba7a0938a271a2e2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.498109 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ee0cde267441f2ba7a0938a271a2e2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.498799 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ee0cde267441f2ba7a0938a271a2e2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.499410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ee0cde267441f2ba7a0938a271a2e2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.512656 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1447/2895] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:34 2022] DELETE /identity/v3/users/f8ee0cde267441f2ba7a0938a271a2e2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:34.793882 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:34.795363 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:34.812804 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7638dc9-fe7a-4dd9-9bfb-f59c14f7c55d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:34.871598 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:34.872370 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:34.872651 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:34.872927 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] PATH_INFO: `/v3/projects/53ce20ef5bcc4a0486762d6591ab5b44` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:34.874161 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ce20ef5bcc4a0486762d6591ab5b44)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:34.876592 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:34.923022 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ce20ef5bcc4a0486762d6591ab5b44', 'request_id': 'req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.929094 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ce20ef5bcc4a0486762d6591ab5b44', 'request_id': 'req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.934546 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ce20ef5bcc4a0486762d6591ab5b44', 'request_id': 'req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.948292 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] The token cache is being invalidate because project 53ce20ef5bcc4a0486762d6591ab5b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:34.948580 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.949209 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:34.950845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1449/2896] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:34 2022] DELETE /identity/v3/projects/53ce20ef5bcc4a0486762d6591ab5b44 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:34.956112 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:34.957580 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:34.971147 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0fe3-eb38-4442-8b79-29de43c21e59 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:35.023016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:35.029804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:35.030721 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.031183 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.031855 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] PATH_INFO: `/v3/users/96eae654e7df4d54850b02e74a1c2f0d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.033402 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=96eae654e7df4d54850b02e74a1c2f0d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:35.035908 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:35.091173 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96eae654e7df4d54850b02e74a1c2f0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.099164 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96eae654e7df4d54850b02e74a1c2f0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.113909 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96eae654e7df4d54850b02e74a1c2f0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.114591 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96eae654e7df4d54850b02e74a1c2f0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.116528 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8847ce6d-83c0-40d3-b8ea-144997be33bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96eae654e7df4d54850b02e74a1c2f0d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.130288 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1448/2897] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:34 2022] DELETE /identity/v3/users/96eae654e7df4d54850b02e74a1c2f0d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:35.446442 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:35.448561 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:35.463834 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0ce5e-b0a5-4719-afcb-462a1de2d88f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:35.509513 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:35.528396 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:35.529332 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.529620 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.529898 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] PATH_INFO: `/v3/projects/4a982f70e39b4484b1dc568cd9318683` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.531263 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a982f70e39b4484b1dc568cd9318683)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:35.533835 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:35.583825 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a982f70e39b4484b1dc568cd9318683', 'request_id': 'req-67152b1e-a186-4b70-8408-5716ac164950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.590015 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a982f70e39b4484b1dc568cd9318683', 'request_id': 'req-67152b1e-a186-4b70-8408-5716ac164950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.596376 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a982f70e39b4484b1dc568cd9318683', 'request_id': 'req-67152b1e-a186-4b70-8408-5716ac164950'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.610928 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] The token cache is being invalidate because project 4a982f70e39b4484b1dc568cd9318683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:35.611213 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.611845 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-67152b1e-a186-4b70-8408-5716ac164950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:35.613432 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1450/2898] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:35 2022] DELETE /identity/v3/projects/4a982f70e39b4484b1dc568cd9318683 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:35.618881 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09546b-d359-4472-8e49-d6146f2ebe62 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.619245 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09546b-d359-4472-8e49-d6146f2ebe62 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.619513 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09546b-d359-4472-8e49-d6146f2ebe62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.640332 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-de09546b-d359-4472-8e49-d6146f2ebe62 None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:35.642124 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-de09546b-d359-4472-8e49-d6146f2ebe62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:35.705195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1449/2899] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:35.713821 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3811456-e2a8-42e2-94dd-c5859e7ac35e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.714111 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3811456-e2a8-42e2-94dd-c5859e7ac35e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.714384 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3811456-e2a8-42e2-94dd-c5859e7ac35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.734167 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d3811456-e2a8-42e2-94dd-c5859e7ac35e None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:35.735429 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d3811456-e2a8-42e2-94dd-c5859e7ac35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:35.743439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1451/2900] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:35.747740 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09546b-d359-4472-8e49-d6146f2ebe62 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:35.758353 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09546b-d359-4472-8e49-d6146f2ebe62 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:35.763365 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:35.764047 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.764334 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.764613 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.765399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:12:35.765713 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:35.767922 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:35.780054 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1450/2901] 10.222.0.91 () {58 vars in 1152 bytes} [Thu May 26 14:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:12:35.785722 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3811456-e2a8-42e2-94dd-c5859e7ac35e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:35.795156 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3811456-e2a8-42e2-94dd-c5859e7ac35e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:35.799859 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:35.800584 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.800876 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.801180 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.801884 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:35.803770 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:35.830032 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1452/2902] 10.222.0.91 () {60 vars in 1152 bytes} [Thu May 26 14:12:35 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:35.834196 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:35.844063 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aabc38-0ed7-49de-af67-eb5055b45803 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:35.899314 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:35.899902 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.900187 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.900463 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.901184 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:35.903093 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:35.912146 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:35.927111 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1451/2903] 10.222.0.91 () {60 vars in 1143 bytes} [Thu May 26 14:12:35 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:12:35.931052 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:35.941475 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365d57f-4317-4c60-9d66-819420abf6a5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:35.946334 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:35.946996 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.947275 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.947553 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.948274 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:35.949911 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:35.961070 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1453/2904] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:35.965886 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:35.978413 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e3471-b172-447c-a8c4-442c3ebdf664 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:35.983508 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd7bbb-6db4-48f0-ae44-7d177eab2d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:35.984195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd7bbb-6db4-48f0-ae44-7d177eab2d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:35.984479 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd7bbb-6db4-48f0-ae44-7d177eab2d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:35.984784 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd7bbb-6db4-48f0-ae44-7d177eab2d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:35.985604 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd7bbb-6db4-48f0-ae44-7d177eab2d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:35.987410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd7bbb-6db4-48f0-ae44-7d177eab2d62 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:35.998016 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1452/2905] 10.222.0.91 () {58 vars in 1121 bytes} [Thu May 26 14:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:12:36.002709 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:36.015016 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261ca67-6afc-40d4-9815-89d63b65f60f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:36.020567 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:36.021534 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:36.021828 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:36.022107 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] PATH_INFO: `/v3/projects/5e33e2163a504c1f93d8f64c6fceb3d6/users/bdc772ae3691486780fd408c7b2d35b4/roles/8719b7bceaea49a48ff010ff8d107832` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:36.038813 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e33e2163a504c1f93d8f64c6fceb3d6, user_id=bdc772ae3691486780fd408c7b2d35b4, role_id=8719b7bceaea49a48ff010ff8d107832)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:36.041979 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:36.055424 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1454/2906] 10.222.0.91 () {60 vars in 1482 bytes} [Thu May 26 14:12:36 2022] PUT /identity/v3/projects/5e33e2163a504c1f93d8f64c6fceb3d6/users/bdc772ae3691486780fd408c7b2d35b4/roles/8719b7bceaea49a48ff010ff8d107832 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:36.066808 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e02b-d9bd-425b-9f3c-4769bb969a28 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:36.067097 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e02b-d9bd-425b-9f3c-4769bb969a28 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:36.067394 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e02b-d9bd-425b-9f3c-4769bb969a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:36.094233 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d9e8e02b-d9bd-425b-9f3c-4769bb969a28 None None] MFA Rules not processed for user `bdc772ae3691486780fd408c7b2d35b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:36.095462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d9e8e02b-d9bd-425b-9f3c-4769bb969a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:36.164887 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1453/2907] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:12:36.177705 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:36.179525 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:36.245889 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af71422-6468-405d-a04b-74d1095e6c47 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:36.249832 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95b69f-4b22-4d26-ad40-a7796ee562ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a381dccf1ca4dacaab4e0ae0d473525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:36.250556 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95b69f-4b22-4d26-ad40-a7796ee562ca service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:36.250841 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95b69f-4b22-4d26-ad40-a7796ee562ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:36.251659 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95b69f-4b22-4d26-ad40-a7796ee562ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:36.262027 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95b69f-4b22-4d26-ad40-a7796ee562ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:36.263488 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95b69f-4b22-4d26-ad40-a7796ee562ca service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:36.273634 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1455/2908] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:44.354343 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e02b-d9bd-425b-9f3c-4769bb969a28 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:44.417366 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e02b-d9bd-425b-9f3c-4769bb969a28 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:44.421851 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:44.422742 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:44.423020 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:44.423289 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] PATH_INFO: `/v3/users/bdc772ae3691486780fd408c7b2d35b4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:44.424462 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc772ae3691486780fd408c7b2d35b4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:44.426399 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:44.481665 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc772ae3691486780fd408c7b2d35b4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:44.488943 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc772ae3691486780fd408c7b2d35b4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:44.502544 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc772ae3691486780fd408c7b2d35b4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:44.503243 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc772ae3691486780fd408c7b2d35b4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:44.503867 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc772ae3691486780fd408c7b2d35b4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:44.506278 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:44.524098 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1454/2909] 10.222.0.91 () {58 vars in 1223 bytes} [Thu May 26 14:12:44 2022] DELETE /identity/v3/users/bdc772ae3691486780fd408c7b2d35b4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:44.532367 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:44.532948 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:44.533042 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:44.550626 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] MFA Rules not processed for user `42b8c6ab6fbb49deaec22b8b6964d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:12:44.551862 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:44.558315 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:44.632519 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1456/2910] 10.222.0.91 () {58 vars in 957 bytes} [Thu May 26 14:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:12:44.642835 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:44.644410 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:44.705883 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3d5ec-81f7-498b-8ffb-e54c7347c4b4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:44.709619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79d624b7de50480bb4002eb23ec2b360', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:44.710271 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:44.710554 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:44.710831 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:44.720123 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:44.721586 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:44.731046 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1455/2911] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:12:45.002435 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:12:45.003983 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:12:45.016515 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e3773-536d-4dce-9e11-800c78187dcf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:12:45.021800 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b8c6ab6fbb49deaec22b8b6964d6a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e741295126a84576b8368706b7ffe76d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:12:45.022676 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:12:45.022986 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:12:45.023261 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] PATH_INFO: `/v3/projects/5e33e2163a504c1f93d8f64c6fceb3d6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:12:45.024463 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e33e2163a504c1f93d8f64c6fceb3d6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:12:45.026439 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:12:45.074975 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e33e2163a504c1f93d8f64c6fceb3d6', 'request_id': 'req-41c9074e-37ed-41f1-a384-9457785bbf29'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:45.081599 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e33e2163a504c1f93d8f64c6fceb3d6', 'request_id': 'req-41c9074e-37ed-41f1-a384-9457785bbf29'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:45.088103 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e33e2163a504c1f93d8f64c6fceb3d6', 'request_id': 'req-41c9074e-37ed-41f1-a384-9457785bbf29'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:45.101105 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] The token cache is being invalidate because project 5e33e2163a504c1f93d8f64c6fceb3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:12:45.101421 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:45.102044 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-41c9074e-37ed-41f1-a384-9457785bbf29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:12:45.103603 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1457/2912] 10.222.0.91 () {58 vars in 1232 bytes} [Thu May 26 14:12:45 2022] DELETE /identity/v3/projects/5e33e2163a504c1f93d8f64c6fceb3d6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:12:54.726804 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=30 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:12:55.098010 dv-cinder-842130-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:15:26.077782 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:15:26.079619 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:15:26.162405 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782dd22-9ac2-4789-9945-41cd9234b07f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:15:26.233533 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39a03a-4929-45bf-ac93-2d000da1a634 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997348872c3241f4a14c6b2fe9b66211', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '061d1d45f0894be3b0165e8360b0035c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:15:26.234352 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39a03a-4929-45bf-ac93-2d000da1a634 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:15:26.234717 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39a03a-4929-45bf-ac93-2d000da1a634 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:15:26.235197 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39a03a-4929-45bf-ac93-2d000da1a634 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:15:26.237746 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ab39a03a-4929-45bf-ac93-2d000da1a634 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:15:26.254195 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39a03a-4929-45bf-ac93-2d000da1a634 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:15:26.256418 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39a03a-4929-45bf-ac93-2d000da1a634 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:15:26.270872 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1456/2913] 10.222.0.91 () {60 vars in 1365 bytes} [Thu May 26 14:15:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:15:36.264718 dv-cinder-842130-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=58 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}